Sorry, I am back now. It seems that my setup had complicated things: I was trying stuff on a laptop while the actual setup was on another host (with the actual GPU), but I did use the correct hostname for the workstation, which should (I mean, that is the whole point of Nix?) lead to the same build. (Both systems are x86_64) However I was also trying globally enable cudaSupport and package-overridden cudaSupport, which might have lead to me making a mistake, I don't know. All I can say is that

nix build --override-input nixpkgs github:NixOS/nixpkgs/9f41a78ead0fbe2197cd4c09b5628060456cd6e3 .#nixosConfigurations.$(hostname).pkgs.onnxruntime

now just downloads onnxruntime from the cache, which is the expected behaviour. I'm going to check without overridden input and pytorch again and then with the whole system.
Another question though: How would I override cudaSupport for a single package and its dependencies? Like llama-cpp is easy (llama-cpp.package = pkgs.override { cudaSupport = true; }) but open-webui itself does not have a cudaSupport option, but onnxruntime and pytorch do.

Another question though: How would I override cudaSupport for a single package and its dependencies?

That's why we use import nixpkgs { config.cudaSupport = true; }

In reply to @ruroruro:matrix.org
Unless I am misunderstanding something, looking at the Southern California Linux Expo youtube channel, it seems that the only nix related streams from the 6th and 7th of march are for Room 101. And the "Evaluating the Nix Evaluator" talk was scheduled for Room 2 (which I am assuming is actually Room 102)? I wonder...

RE: https://discourse.nixos.org/t/why-does-nixpkgs-config-exist/61456/6
RE: wrappers, stdenv, etc.
RE: "The easiest way to achieve consistency today is to just enable cudaSupport Nixpkgs-wide using config"

And yet again, I don't think it has to be this way. It's not inherent, it's just a trait of callPackage/makeScope and of overrides being local. To try and defend the notion of the hypothetical stdenv.cudaSupport, if all of a package's inputs consistently accepted stdenv as a parameter, and could be entirely configured just using stdenv, then all it'd take to achieve consistency is to apply .override { inherit stdenv; } to all *Inputs. And if we didn't limit ourselves to the idea of callPackage and that there's a global "container" from which we "fill in" our parameters, that could probably be automated (by doing dynamic scoping essentially)

But ok let's leave callPackage and scoping aside. This still made me think of how to layer nvcc-wrapper and the conditional stdenv functionality. I'm back to the notion that we actually can move cudaSupport to (the platform attrsets and) stdenv, replace optionals cudaSupport with optionals stdenv.cudaSupport, and implement cudaStdenv.mkDerivation in such a way that it would be able to downgrade the compiler if it detects that its argument is capable of using CUDA functionality. The detection could still be done in two ways: we could demand that derivations explicitly pass an argument such as, say, cudaCapable = true, or we could detect that nvcc is in its nativeBuildInputs. I'd vote for the former because llvm, nvc++, and it's more explicit too. The cudaStdenv constructor could be made configurable so that it's possible to disable the downgrading (e.g. when we don't care about consistency or the LTO and we just trust in the wrapper; or if llvm offers more relaxed constraints). And regardless of the stdenv layer, the lower-level nvcc wrapper can exist and be used independently.

CC connor (he/him) (UTC-8)

I just wanted to make sure. You commented

H'm well it seems to make sense at least for

nix eval nixpkgs#cudaPackages.cudnn_8_0.brokenConditions
{ "CUDA version is too new" = true; ... }

Did I understand correctly, that you are okay with

• moving cudnn.brokenConditions."CUDA version is too old/new" to badPlatformsConditions
• moving nsight_systems.brokenConditions."CUDA too old (<11.8)" to badPlatformsConditions
• moving nvidia_driver.brokenConditions."Package is not supported; use drivers from linuxPackages" to badPlatformsConditions
• adding tensorrt.hydraPlatforms = none
  ?

I just wanted to make sure. You commented

H'm well it seems to make sense at least for

nix eval nixpkgs#cudaPackages.cudnn_8_0.brokenConditions
{ "CUDA version is too new" = true; ... }

Did I understand correctly, that you are okay with

• moving cudnn.brokenConditions."CUDA version is too old/new" to badPlatformsConditions
• moving nsight_systems.brokenConditions."CUDA too old (<11.8)" to badPlatformsConditions
• moving nvidia_driver.brokenConditions."Package is not supported; use drivers from linuxPackages" to badPlatformsConditions
• adding tensorrt.hydraPlatforms = none

?

I just wanted to make sure. You left this comment on the release-cuda issue:

H'm well it seems to make sense at least for

nix eval nixpkgs#cudaPackages.cudnn_8_0.brokenConditions
{ "CUDA version is too new" = true; ... }

Did I understand correctly, that you are okay with

• moving cudnn.brokenConditions."CUDA version is too old/new" to badPlatformsConditions
• moving nsight_systems.brokenConditions."CUDA too old (<11.8)" to badPlatformsConditions
• moving nvidia_driver.brokenConditions."Package is not supported; use drivers from linuxPackages" to badPlatformsConditions
• adding tensorrt.hydraPlatforms = none

?

I just wanted to make sure. You left this comment on the release-cuda issue:

H'm well it seems to make sense at least for

nix eval nixpkgs#cudaPackages.cudnn_8_0.brokenConditions
{ "CUDA version is too new" = true; ... }

Did I understand correctly, that you are okay with

• moving cudnn.brokenConditions."CUDA version is too old/new" to badPlatformsConditions
• moving nsight_systems.brokenConditions."CUDA too old (<11.8)" to badPlatformsConditions
• moving nvidia_driver.brokenConditions."Package is not supported; use drivers from linuxPackages" to badPlatformsConditions
• adding tensorrt.hydraPlatforms = none

?

As I've mentioned, you can see the exact code I am talking about in this branch.

I just wanted to make sure. You left this comment on the release-cuda issue:

H'm well it seems to make sense at least for

nix eval nixpkgs#cudaPackages.cudnn_8_0.brokenConditions
{ "CUDA version is too new" = true; ... }

Did I understand correctly, that you are okay with

• moving cudnn.brokenConditions."CUDA version is too old/new" to badPlatformsConditions
• moving nsight_systems.brokenConditions."CUDA too old (<11.8)" to badPlatformsConditions
• moving nvidia_driver.brokenConditions."Package is not supported; use drivers from linuxPackages" to badPlatformsConditions
• adding tensorrt.hydraPlatforms = none

?

As I've mentioned, you can see the exact code I am talking about in here.

I just wanted to make sure. You left this comment on the release-cuda issue:

H'm well it seems to make sense at least for

nix eval nixpkgs#cudaPackages.cudnn_8_0.brokenConditions
{ "CUDA version is too new" = true; ... }

Did I understand correctly, that you are okay with

• moving cudnn.brokenConditions."CUDA version is too old/new" to badPlatformsConditions
• moving nsight_systems.brokenConditions."CUDA too old (<11.8)" to badPlatformsConditions
• moving nvidia_driver.brokenConditions."Package is not supported; use drivers from linuxPackages" to badPlatformsConditions
• adding tensorrt.hydraPlatforms = none

?

As I've mentioned, you can see the exact code I am talking about here.