I am on x86_64, nixos-unstable with flakes with an RTX 3060 Ti and following substituters:

        substituters = [
          "https://nix-community.cachix.org"
        ];
        trusted-public-keys = [
          "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
        ];

I am definitely building onnxruntime myself even though I get:

> nix path-info --override-input nixpkgs github:NixOS/nixpkgs/9f41a78ead0fbe2197cd4c09b5628060456cd6e3 --derivation .\#nixosConfigurations.$(hostname).pkgs.onnxruntime
• Updated input 'nixpkgs':
    'github:nixos/nixpkgs/32fb99ba93fea2798be0e997ea331dd78167f814?narHash=sha256-ozoOtE2hGsqh4XkTJFsrTkNxkRgShxpQxDynaPZUGxk%3D' (2025-02-21)
  → 'github:NixOS/nixpkgs/9f41a78ead0fbe2197cd4c09b5628060456cd6e3?narHash=sha256-WWXRCTOWcKvtzqzVgBMON0/TWcFMyWq831HQUITE4rs%3D' (2025-02-21)
/nix/store/a22vqi9d0ndhlcy1yxw4m3ir4z7ckfrg-onnxruntime-1.20.1.drv

Which is the same hash as the hydra build store path

Sorry, I am back now. It seems that my setup had complicated things: I was trying stuff on a laptop while the actual setup was on another host (with the actual GPU), but I did use the correct hostname for the workstation, which should (I mean, that is the whole point of Nix?) lead to the same build. (Both systems are x86_64) However I was also trying globally enable cudaSupport and package-overridden cudaSupport, which might have lead to me making a mistake, I don't know. All I can say is that

nix build --override-input nixpkgs github:NixOS/nixpkgs/9f41a78ead0fbe2197cd4c09b5628060456cd6e3 .#nixosConfigurations.$(hostname).pkgs.onnxruntime

now just downloads onnxruntime from the cache, which is the expected behaviour. I'm going to check without overridden input and pytorch again and then with the whole system.
Another question though: How would I override cudaSupport for a single package and its dependencies? Like llama-cpp is easy (llama-cpp.package = pkgs.override { cudaSupport = true; }) but open-webui itself does not have a cudaSupport option, but onnxruntime and pytorch do.

Another question though: How would I override cudaSupport for a single package and its dependencies?

That's why we use import nixpkgs { config.cudaSupport = true; }