In reply to @glepage:matrix.org

Looks like rustfmt is broken on x86_64-darwin on nixos-25.05:

https://buildbot.nix-community.org/#/builders/2847/builds/5266/steps/1/logs/stdio (from Nixvim CI)

I think we can agree that fetchCargoVendor being a two step construction is not very good for the cache.
Now, we could move the calling of the "fetch-cargo-vendor-util create-vendor" into cargoSetupHook, but that would create a dependency on fetch-cargo-vendor-util for every rust package.

Now I think I came up with a pretty clever workaround, which keeps the two layers, but makes the non-FOD layer essentially just an empty derivation except a binary that you can run to start the unpacking process.
This way we just have to detect if that unpacking binary is present in cargoDeps inside cargoSetupHook and then call it.

Though, this would make it so that we can't "easily" get derivation with the vendored dependencies unpacked from the tarball.
But, we could provide a passthru.vendorFetched (things in passthru don't get build by CI) which just runs the unpacking logic into "$out"

(Also, I put in a symlink into the non-FOD part for vendorStaging for easier access)

I made a very rudimentary POC: https://github.com/NixOS/nixpkgs/pull/427815

Note: I did not take into consideration cross/splicing

What do you all think?

90% of Rust packages will already have that dependency and it's just a lightweight Python script, yeah?

I'm assuming that some people are using a custom implementation for vendoring. (e.g. for some authentication logic IDK) and their logic assumes that cargoSetupHook expects an unpacked cargo vendor structure.
So I don't think we should unconditionally use fetch-cargo-vendor-util if we detect that cargoDeps is a directory.
We'd have to conditionally run our logic, only when we know we're using something that was generated by fetchCargoVendor
However, we can't really put any distinction marker files into vendor-staging, since that's a FOD.

But maybe I'm overthinking this.

Yes getting rid of the two steps would be healthier for easier overriding of hashes via overrideAttrs

Anyways, I have to go now... I'll be back in an hour or so, sorry!

In reply to @niklaskorz:matrix.org
@Toma I rebased + reran your treewide to resolve the merge conflicts, will also run nixpkgs-review for darwin and linux in a bit, but otherwise I think it’s good to go?

As discussed earlier, we were considering waiting until https://github.com/NixOS/nixpkgs/pull/423228 gets into stable so that we can do this treewide both on stable and master, to avoid much of the backporting issues.
Though, there hasn't been a staging-next-25.05 cycle since then...

So maybe we can deal with a few backporting conflicts until then.

idk, what do you think is better?