| 16 Dec 2024 |
 connor (burnt/out) (UTC-8) | I'd recommend testing the waters and getting a sense of prior art done in terms of extending those; perhaps the folks in the Nixpkgs Stdenv room would be good to reach out to? https://matrix.to/#/#stdenv:nixos.org
After that (and you've gotten a list of people interested in or knowledgeable about such work), I think drafting an RFC would be the next step, to fully lay out the plan for design and implementation. If it's a relatively small change, maybe an RFC is unnecessary and a PR would be fine! | 06:55:30 |
 SomeoneSerge (back on matrix) | That's the reason I've lately been ignoring flakes in personal projects: just stick to impure eval and autocalling. For me it's usually going in the direction of
{ nixpkgs ? <nixpkgs>
, pkgs ? import nixpkgs { inherit config; }
, lib ? pkgs.lib
, cudaSupport ? false
, config ? { inherit cudaSupport; }
}:
lib.makeScope pkgs.newScope (self: {
# ...
})
| 14:45:04 |
| SomeoneSerge (back on matrix) | * That's the reason I've lately been ignoring flakes in personal projects: just stick to impure eval and autocalling. For me it's usually going in the direction of
{ nixpkgs ? <nixpkgs>
, pkgs ? import nixpkgs { inherit config; }
, lib ? pkgs.lib
, cudaSupport ? false
, config ? { inherit cudaSupport; }
}:
lib.makeScope pkgs.newScope (self: {
# ...
})
| 14:45:18 |
 matthewcroughan | I actually ended up with a good pattern using flake.parts | 14:45:28 |
| matthewcroughan | perSystem = { system, ... }: {
_module.args.rocmPkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
(self: super: {
python3 = super.python3.override {
packageOverrides = self: super: { torch = super.torch-bin; };
};
})
];
config.allowUnfree = true;
config.rocmSupport = true;
inherit system;
};
_module.args.nvidiaPkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
];
config.allowUnfree = true;
config.cudaSupport = true;
inherit system;
};
_module.args.pkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
];
config.allowUnfree = true;
inherit system;
};
};
| 14:46:22 |
| matthewcroughan | * perSystem = { system, ... }: {
_module.args.rocmPkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
(self: super: {
python3 = super.python3.override {
packageOverrides = self: super: { torch = super.torch-bin; };
};
})
];
config.allowUnfree = true;
config.rocmSupport = true;
inherit system;
};
_module.args.nvidiaPkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
];
config.allowUnfree = true;
config.cudaSupport = true;
inherit system;
};
_module.args.pkgs = import inputs.nixpkgs {
overlays = [
inputs.self.overlays.default
];
config.allowUnfree = true;
inherit system;
};
};
| 14:46:27 |
| matthewcroughan | Could be deduplicated using a function, but this is what it looks like unfolded | 14:46:38 |
| matthewcroughan | Then other flake-modules have these rocmPkgs and nvidiaPkgs arguments passed to them | 14:47:01 |
| matthewcroughan | perSystem = { config, pkgs, nvidiaPkgs, rocmPkgs, system, ... }: {
packages = {
comfyui-nvidia = nvidiaPkgs.comfyuiPackages.comfyui;
comfyui-amd = rocmPkgs.comfyuiPackages.comfyui;
};
};
| 14:47:04 |
| matthewcroughan | * perSystem = { config, pkgs, nvidiaPkgs, rocmPkgs, system, ... }: {
packages = {
comfyui-nvidia = nvidiaPkgs.comfyuiPackages.comfyui;
comfyui-amd = rocmPkgs.comfyuiPackages.comfyui;
};
};
| 14:47:05 |
| SomeoneSerge (back on matrix) | I think this should be doable without major backwards incompatible changes? | 14:47:07 |
| matthewcroughan | the issue is that rocmSupport is fully broken in Nixpkgs, so this doesn't work, but it should | 14:47:25 |
| matthewcroughan | * the issue is that rocmSupport is fully broken in Nixpkgs, so this doesn't work, but it should in future | 14:47:29 |
| SomeoneSerge (back on matrix) | That's more or less what the llama-cpp flake did, but didn't you say
I don't want to have attributes like nvidia-myapp rocm-myapp and myapp
| 14:48:09 |
| matthewcroughan | I'm happy as long as I don't have to do weird things to achieve it | 14:48:58 |
| matthewcroughan | and for me, this is not weird | 14:49:02 |
| matthewcroughan | previously what my flake was doing was far weirder | 14:49:07 |
| matthewcroughan | https://github.com/nixified-ai/flake/blob/master/projects/invokeai/default.nix#L66-L96 | 14:49:32 |
| matthewcroughan | previously it was defining functions that were able to create variants of packages without setting rocmSupport or cudaSupport | 14:49:51 |
| matthewcroughan | Just terrible | 14:50:00 |
| matthewcroughan | Besides, the modules the flake will export, won't interact with the comfyui-nvidia or comfyui-amd attrs, this is just for people who want to try it with nix run | 14:50:42 |
| matthewcroughan | In a system using the nixosModules, the overlay will be applied, which strictly ignores the packages attr of the flake | 14:51:05 |
| matthewcroughan | the packages attr of the flake is just there for people wanting to use things in a non-nixos context really | 14:53:53 |
| SomeoneSerge (back on matrix) | They're just completely separate.
I guess there are mappings between subsets of the frameworks, as evidenced by ZLUDA, hipify, and https://docs.scale-lang.com.
I suppose one could say that ZLUDA is a sort of a runtime proxy, although the multi-versioning bit is still missing. | 14:55:59 |
| matthewcroughan | Interestingly in the case of comfyui, I didn't need to add any rocm specific stuff | 15:33:35 |
| matthewcroughan | * Interestingly in the case of comfyui, I didn't need to add any rocm or cuda specific stuff | 15:33:37 |
| matthewcroughan | that's all in the deps | 15:33:40 |
| matthewcroughan | So for it, all I do is swap rocmSupport/cudaSupport in the nixpkgs instance, which isgreat | 15:34:35 |
| matthewcroughan | * So for it, all I do is swap rocmSupport/cudaSupport in the nixpkgs instance, which is great | 15:34:37 |
| 17 Dec 2024 |
| connor (burnt/out) (UTC-8) | just left some comments, looks good!
Since you're the first person I'm aware of other than myself to use my cuda-packages repo, I'd love any thoughts you had on the user experience... especially given I've not documented anything yet. | 07:34:41 |
