| 14 Dec 2024 |
 sielicki | matthewcroughan: IMO it might be faster and better for you to write the missing pyproject.toml it needs | 20:34:25 |
 matthewcroughan | Would I not have to rewrite that each and every single time the owner updates the package? | 20:34:49 |
| matthewcroughan | I would rather write down the missing source/destination, instead of filling in for what the developer isn't doing | 20:34:59 |
| matthewcroughan | Does the pyproject.toml actually account for mutable vs immutable ? | 20:35:44 |
| matthewcroughan | If it's a minimal enough difference then maybe I can submit it upstream and convince them to maintain it | 20:36:25 |
| sielicki | although python is interpretted you still end up with an automatic installation phase that generates bytecode -- it really shouldn't be the case that it needs any access to .py files at runtime | 20:36:59 |
| matthewcroughan | But it does, and this is a pattern present in plenty of projects, and this also happens in PHP all the time | 20:37:26 |
| matthewcroughan | So we have to get over it somehow | 20:37:37 |
| matthewcroughan | I think the least evil method are wrappers that fool the application into seeing a FS you want them to see | 20:38:18 |
| matthewcroughan | Though maintaining that with a series of cp/ln in the installPhase of something is pretty annoying | 20:38:38 |
| sielicki | I guess what I'm saying is that I think just copying the source into $out is insufficient and abnormal -- each of these should be a separate python module and the buildPhase should take care of both copying it into expected python path dir structures and byte-compiling it. Yes, it works to just have a py file instead of a pyc file and vice-versa, but you really do want the pyc files | 20:44:58 |
| matthewcroughan |
I guess what I'm saying is that I think just copying the source into $out is insufficient and abnormal
Yes, but abnormal applications exist, and make up the majority of nixpkgs
| 20:45:26 |
| sielicki | as far as it relates to cuda -- as long as you have cudaified torch in the python environment and the shebangs are modified to refer to that python environment, I don't think there's any real concern around what you're doing in the installPhase | 20:47:26 |
| sielicki | just keep in mind the general restrictions around running cuda apps on non-nixos | 20:47:42 |
| matthewcroughan | yeah because the complexities of advertising more than nvidia/cuda outside of nixpkgs are high | 20:50:21 |
| matthewcroughan | even if it reduces closure size, the overhead of writing nix code that is able to provide nvidia/rocm is really a lot | 20:50:49 |
| matthewcroughan | * even if it reduces closure size, the overhead of writing nix code that is able to provide nvidia/rocm separately is really a lot | 20:50:54 |
| matthewcroughan | I don't want to have attributes like nvidia-myapp rocm-myapp and myapp | 20:51:36 |
| matthewcroughan | would be much better to have a single derivation that is capable of all at runtime | 20:52:00 |
| sielicki | you shouldn't have to -- the torch input to your derivation will check whether nixpkgs was imported with config.cudaSupport and transparently do the right thing | 20:52:11 |
| matthewcroughan | yes, which means in my flake that isn't nixpkgs, I will have to import nixpkgs 3 times to handle those 3 cases | 20:52:30 |
| matthewcroughan | and also write nix code three times to handle the three cases | 20:52:51 |
| matthewcroughan | I could delay it by using an overlay and just not create 3 attributes upfront in the packages attribute of a flake | 20:53:22 |
| matthewcroughan | then it will depend on the importer.. but then I can't just have people nix run my attributes, they will have to write their own flake to consume, and specify the support in their own flake (which imo is not ideal) | 20:53:50 |
| sielicki | nix run --impure should check the users ~/.config/nixpkgs file and do the right thing around nonFree and/or cudaSupport/rocmSupport. If you want to add separate imports of nixpkgs for your checks that's one thing but principally I think you should just expose a single package | 20:55:40 |
| matthewcroughan | I'm not sure why I'd want a flake on GitHub that reads from the user's home directory at evaluation time. | 20:58:31 |
| matthewcroughan | I'd just want packages.x86_64-linux.{nvidia-myapp,rocm-myapp} | 20:59:11 |
| matthewcroughan | In theory, is there not a stub we could use to make it gpuSupport = true ? | 21:51:35 |
| matthewcroughan | and infer the required libs for rocm or nvidia, from some intermediate representation? | 21:51:52 |
| matthewcroughan | buildInputs = with gpuPackages; [ somethingCommon ] which would decided what somethingCommon is based on cudaSupport or rocmSupport? | 21:52:26 |
