In reply to @matthewcroughan:defenestrate.it
if you enable cudaSupport and rocmSupport, what happens? Do you actually get an output that is usable for both?

That depends entirely on the project and the nix expression for it.
For the projects I’m aware of (Magma and PyTorch) CUDA and ROCm backends are exclusive of each other and the build systems would require major rewrites to produce a single build with support for both.

From what I understand from what you desire, the closest analogue I can think of would be Apple’s “universal binary” which supports both x86-64 and aarch64… but I suspect you’d also want the equivalent of function multiversioning, where each function is compiled into multiple variants using different architecture features (think SSE, NEON, AVX256, AVX512, etc.) so that at runtime the function matching the host’s most advanced capability is used. This would correspond to CUDA compute capabilities.

NVCC can produce binaries with device code for multiple capabilities, but it does increase the compile time and binary size significantly — enough so that linking can fail due to size limitations! That’s part of the reason Magma in Nixpkgs produces a static library when building with CUDA.

       error: 'miopengemm' has been deprecated.
       It is still available for some time as part of rocmPackages_5.

SomeoneSerge (utc+3): connor (he/him) (UTC-7): I spent some time revamping my personal flake today/yesterday and now have a better understanding of the new hostPlatform/buildPlatform stuff, alongside lib.platforms

I do think it's the right interface long-term for both cudaSupport, gencodes, and also configuring fabrics.

The entire extent of my contributions to nixpkgs has been just doing small contributions to specific packages. I want to write up a small doc proposing this and discussing the migration path, but also wanted to collaborate w/ you guys. I don't even know where to post this, do I just open an issue on gh or does it need to go on the discourse?

That's the reason I've lately been ignoring flakes in personal projects: just stick to impure eval and autocalling. For me it's usually going in the direction of

{ nixpkgs ? <nixpkgs>
, pkgs ? import nixpkgs { inherit config; }
, lib ? pkgs.lib
, cudaSupport ? false
, config ? { inherit cudaSupport; }
}:

lib.makeScope pkgs.newScope (self: {
 # ...
})

That's the reason I've lately been ignoring flakes in personal projects: just stick to impure eval and autocalling. For me it's usually going in the direction of

{ nixpkgs ? <nixpkgs>
, pkgs ? import nixpkgs { inherit config; }
, lib ? pkgs.lib
, cudaSupport ? false
, config ? { inherit cudaSupport; }
}:

lib.makeScope pkgs.newScope (self: {
 # ...
})

      perSystem = { system, ... }: {
        _module.args.rocmPkgs = import inputs.nixpkgs {
          overlays = [
            inputs.self.overlays.default
            (self: super: {
              python3 = super.python3.override {
                packageOverrides = self: super: { torch = super.torch-bin; };
              };
            })
          ];
          config.allowUnfree = true;
          config.rocmSupport = true;
          inherit system;
        };
        _module.args.nvidiaPkgs = import inputs.nixpkgs {
          overlays = [
            inputs.self.overlays.default
          ];
          config.allowUnfree = true;
          config.cudaSupport = true;
          inherit system;
        };
        _module.args.pkgs = import inputs.nixpkgs {
          overlays = [
            inputs.self.overlays.default
          ];
          config.allowUnfree = true;
          inherit system;
        };
      };

      perSystem = { system, ... }: {
        _module.args.rocmPkgs = import inputs.nixpkgs {
          overlays = [
            inputs.self.overlays.default
            (self: super: {
              python3 = super.python3.override {
                packageOverrides = self: super: { torch = super.torch-bin; };
              };
            })
          ];
          config.allowUnfree = true;
          config.rocmSupport = true;
          inherit system;
        };
        _module.args.nvidiaPkgs = import inputs.nixpkgs {
          overlays = [
            inputs.self.overlays.default
          ];
          config.allowUnfree = true;
          config.cudaSupport = true;
          inherit system;
        };
        _module.args.pkgs = import inputs.nixpkgs {
          overlays = [
            inputs.self.overlays.default
          ];
          config.allowUnfree = true;
          inherit system;
        };
      };

  perSystem = { config, pkgs, nvidiaPkgs, rocmPkgs, system, ... }: {
    packages = {
      comfyui-nvidia = nvidiaPkgs.comfyuiPackages.comfyui;
      comfyui-amd = rocmPkgs.comfyuiPackages.comfyui;
    };
  };

  perSystem = { config, pkgs, nvidiaPkgs, rocmPkgs, system, ... }: {
    packages = {
      comfyui-nvidia = nvidiaPkgs.comfyuiPackages.comfyui;
      comfyui-amd = rocmPkgs.comfyuiPackages.comfyui;
    };
  };

That's more or less what the llama-cpp flake did, but didn't you say

I don't want to have attributes like nvidia-myapp rocm-myapp and myapp