| 14 Dec 2024 |
 matthewcroughan | Does the pyproject.toml actually account for mutable vs immutable ? | 20:35:44 |
| matthewcroughan | If it's a minimal enough difference then maybe I can submit it upstream and convince them to maintain it | 20:36:25 |
 sielicki | although python is interpretted you still end up with an automatic installation phase that generates bytecode -- it really shouldn't be the case that it needs any access to .py files at runtime | 20:36:59 |
| matthewcroughan | But it does, and this is a pattern present in plenty of projects, and this also happens in PHP all the time | 20:37:26 |
| matthewcroughan | So we have to get over it somehow | 20:37:37 |
| matthewcroughan | I think the least evil method are wrappers that fool the application into seeing a FS you want them to see | 20:38:18 |
| matthewcroughan | Though maintaining that with a series of cp/ln in the installPhase of something is pretty annoying | 20:38:38 |
| sielicki | I guess what I'm saying is that I think just copying the source into $out is insufficient and abnormal -- each of these should be a separate python module and the buildPhase should take care of both copying it into expected python path dir structures and byte-compiling it. Yes, it works to just have a py file instead of a pyc file and vice-versa, but you really do want the pyc files | 20:44:58 |
| matthewcroughan |
I guess what I'm saying is that I think just copying the source into $out is insufficient and abnormal
Yes, but abnormal applications exist, and make up the majority of nixpkgs
| 20:45:26 |
| sielicki | as far as it relates to cuda -- as long as you have cudaified torch in the python environment and the shebangs are modified to refer to that python environment, I don't think there's any real concern around what you're doing in the installPhase | 20:47:26 |
| sielicki | just keep in mind the general restrictions around running cuda apps on non-nixos | 20:47:42 |
| matthewcroughan | yeah because the complexities of advertising more than nvidia/cuda outside of nixpkgs are high | 20:50:21 |
| matthewcroughan | even if it reduces closure size, the overhead of writing nix code that is able to provide nvidia/rocm is really a lot | 20:50:49 |
| matthewcroughan | * even if it reduces closure size, the overhead of writing nix code that is able to provide nvidia/rocm separately is really a lot | 20:50:54 |
| matthewcroughan | I don't want to have attributes like nvidia-myapp rocm-myapp and myapp | 20:51:36 |
| matthewcroughan | would be much better to have a single derivation that is capable of all at runtime | 20:52:00 |
| sielicki | you shouldn't have to -- the torch input to your derivation will check whether nixpkgs was imported with config.cudaSupport and transparently do the right thing | 20:52:11 |
| matthewcroughan | yes, which means in my flake that isn't nixpkgs, I will have to import nixpkgs 3 times to handle those 3 cases | 20:52:30 |
| matthewcroughan | and also write nix code three times to handle the three cases | 20:52:51 |
| matthewcroughan | I could delay it by using an overlay and just not create 3 attributes upfront in the packages attribute of a flake | 20:53:22 |
| matthewcroughan | then it will depend on the importer.. but then I can't just have people nix run my attributes, they will have to write their own flake to consume, and specify the support in their own flake (which imo is not ideal) | 20:53:50 |
| sielicki | nix run --impure should check the users ~/.config/nixpkgs file and do the right thing around nonFree and/or cudaSupport/rocmSupport. If you want to add separate imports of nixpkgs for your checks that's one thing but principally I think you should just expose a single package | 20:55:40 |
| matthewcroughan | I'm not sure why I'd want a flake on GitHub that reads from the user's home directory at evaluation time. | 20:58:31 |
| matthewcroughan | I'd just want packages.x86_64-linux.{nvidia-myapp,rocm-myapp} | 20:59:11 |
| matthewcroughan | In theory, is there not a stub we could use to make it gpuSupport = true ? | 21:51:35 |
| matthewcroughan | and infer the required libs for rocm or nvidia, from some intermediate representation? | 21:51:52 |
| matthewcroughan | buildInputs = with gpuPackages; [ somethingCommon ] which would decided what somethingCommon is based on cudaSupport or rocmSupport? | 21:52:26 |
| matthewcroughan | Are the ecosystems really varying enough to need these two cases? Or can we commonise them? | 21:52:44 |
| 15 Dec 2024 |
 connor (he/him) | I don’t believe there’s a way we can commonise them, at least currently, but my focus has been more on tools (ONNX, PyTorch, etc.) than things that use them (actual models, inference capabilities, etc.), so that’s definitely shaped my opinion.
As I understand it, from an implementation perspective, you’d need some sort of mapping from functionality (BLAS, FFT, DNN) to the actual library (or libraries?) which implement that functionality. But the different ecosystems offer different functionalities and might not have corresponding libraries. | 05:54:52 |
| connor (he/him) | In reply to @matthewcroughan:defenestrate.it
if you enable cudaSupport and rocmSupport, what happens? Do you actually get an output that is usable for both? That depends entirely on the project and the nix expression for it.
For the projects I’m aware of (Magma and PyTorch) CUDA and ROCm backends are exclusive of each other and the build systems would require major rewrites to produce a single build with support for both.
From what I understand from what you desire, the closest analogue I can think of would be Apple’s “universal binary” which supports both x86-64 and aarch64… but I suspect you’d also want the equivalent of function multiversioning, where each function is compiled into multiple variants using different architecture features (think SSE, NEON, AVX256, AVX512, etc.) so that at runtime the function matching the host’s most advanced capability is used. This would correspond to CUDA compute capabilities.
NVCC can produce binaries with device code for multiple capabilities, but it does increase the compile time and binary size significantly — enough so that linking can fail due to size limitations! That’s part of the reason Magma in Nixpkgs produces a static library when building with CUDA. | 06:04:37 |
