| 16 Dec 2024 |
 matthewcroughan | perSystem = { system, ... }: {
_module.args.rocmPkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
(self: super: {
python3 = super.python3.override {
packageOverrides = self: super: { torch = super.torch-bin; };
};
})
];
config.allowUnfree = true;
config.rocmSupport = true;
inherit system;
};
_module.args.nvidiaPkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
];
config.allowUnfree = true;
config.cudaSupport = true;
inherit system;
};
_module.args.pkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
];
config.allowUnfree = true;
inherit system;
};
};
| 14:46:22 |
| matthewcroughan | * perSystem = { system, ... }: {
_module.args.rocmPkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
(self: super: {
python3 = super.python3.override {
packageOverrides = self: super: { torch = super.torch-bin; };
};
})
];
config.allowUnfree = true;
config.rocmSupport = true;
inherit system;
};
_module.args.nvidiaPkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
];
config.allowUnfree = true;
config.cudaSupport = true;
inherit system;
};
_module.args.pkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
];
config.allowUnfree = true;
inherit system;
};
};
| 14:46:27 |
| matthewcroughan | Could be deduplicated using a function, but this is what it looks like unfolded | 14:46:38 |
| matthewcroughan | Then other flake-modules have these rocmPkgs and nvidiaPkgs arguments passed to them | 14:47:01 |
| matthewcroughan | perSystem = { config, pkgs, nvidiaPkgs, rocmPkgs, system, ... }: {
packages = {
comfyui-nvidia = nvidiaPkgs.comfyuiPackages.comfyui;
comfyui-amd = rocmPkgs.comfyuiPackages.comfyui;
};
};
| 14:47:04 |
| matthewcroughan | * perSystem = { config, pkgs, nvidiaPkgs, rocmPkgs, system, ... }: {
packages = {
comfyui-nvidia = nvidiaPkgs.comfyuiPackages.comfyui;
comfyui-amd = rocmPkgs.comfyuiPackages.comfyui;
};
};
| 14:47:05 |
 SomeoneSerge (back on matrix) | I think this should be doable without major backwards incompatible changes? | 14:47:07 |
| matthewcroughan | the issue is that rocmSupport is fully broken in Nixpkgs, so this doesn't work, but it should | 14:47:25 |
| matthewcroughan | * the issue is that rocmSupport is fully broken in Nixpkgs, so this doesn't work, but it should in future | 14:47:29 |
| SomeoneSerge (back on matrix) | That's more or less what the llama-cpp flake did, but didn't you say
I don't want to have attributes like nvidia-myapp rocm-myapp and myapp
| 14:48:09 |
| matthewcroughan | I'm happy as long as I don't have to do weird things to achieve it | 14:48:58 |
| matthewcroughan | and for me, this is not weird | 14:49:02 |
| matthewcroughan | previously what my flake was doing was far weirder | 14:49:07 |
| matthewcroughan | https://github.com/nixified-ai/flake/blob/master/projects/invokeai/default.nix#L66-L96 | 14:49:32 |
| matthewcroughan | previously it was defining functions that were able to create variants of packages without setting rocmSupport or cudaSupport | 14:49:51 |
| matthewcroughan | Just terrible | 14:50:00 |
| matthewcroughan | Besides, the modules the flake will export, won't interact with the comfyui-nvidia or comfyui-amd attrs, this is just for people who want to try it with nix run | 14:50:42 |
| matthewcroughan | In a system using the nixosModules, the overlay will be applied, which strictly ignores the packages attr of the flake | 14:51:05 |
| matthewcroughan | the packages attr of the flake is just there for people wanting to use things in a non-nixos context really | 14:53:53 |
| SomeoneSerge (back on matrix) | They're just completely separate.
I guess there are mappings between subsets of the frameworks, as evidenced by ZLUDA, hipify, and https://docs.scale-lang.com.
I suppose one could say that ZLUDA is a sort of a runtime proxy, although the multi-versioning bit is still missing. | 14:55:59 |
| matthewcroughan | Interestingly in the case of comfyui, I didn't need to add any rocm specific stuff | 15:33:35 |
| matthewcroughan | * Interestingly in the case of comfyui, I didn't need to add any rocm or cuda specific stuff | 15:33:37 |
| matthewcroughan | that's all in the deps | 15:33:40 |
| matthewcroughan | So for it, all I do is swap rocmSupport/cudaSupport in the nixpkgs instance, which isgreat | 15:34:35 |
| matthewcroughan | * So for it, all I do is swap rocmSupport/cudaSupport in the nixpkgs instance, which is great | 15:34:37 |
| 17 Dec 2024 |
 connor (he/him) | just left some comments, looks good!
Since you're the first person I'm aware of other than myself to use my cuda-packages repo, I'd love any thoughts you had on the user experience... especially given I've not documented anything yet. | 07:34:41 |
| connor (he/him) | My understanding was that changes which introduce new functionality or information to ubiquitous components in Nixpkgs should/need to go through the RFC process because people can/do expect stability around those interfaces and so the review process helps find and fix issues with designs before they're implemented.
If there are actual guidelines for changes that would require an RFC (I've not searched hard for them) I'd like to see them so I'm at least aware of them lol | 07:37:58 |
| connor (he/him) | For awareness, I tagged this issue as CUDA related so it should be on our project board: https://github.com/NixOS/nixpkgs/issues/365262 | 07:39:57 |
| 18 Dec 2024 |
|  @dmiskovic:matrix.org joined the room. | 19:38:04 |
| 19 Dec 2024 |
 hexa | https://www.cnx-software.com/2024/12/18/249-nvidia-jetson-orin-nano-super-developer-kit-targets-generative-ai-applications-at-the-edge/ | 17:22:18 |
