Nix Hackers - Public Room Timeline - Matrix Static

	Nix Hackers	912 Members
	For people hacking on the Nix package manager itself	192 Servers

You have reached the beginning of time (for this room).

Sender	Message	Time
1 Jul 2025
roberth	Yes, but only the things you bring into the derivation. You could use a source filter to avoid some unneeded stuff. Currently that's all moot because `fetchTree` copies everything it could return, but we'll change that, and make source filtering a solution for this problem.	11:49:19
magic_rb	In reply to @roberthensing:matrix.org Yes, but only the things you bring into the derivation. You could use a source filter to avoid some unneeded stuff. Currently that's all moot because `fetchTree` copies everything it could return, but we'll change that, and make source filtering a solution for this problem. Right which is why im saying that i think each annexed file should become its own store path, so that youre not copying about 40GB for each build	11:50:14
roberth	If it's not clear which is better, we could make this behavior configurable. Making the right parts of `.git/annex` available to derivations would be a pain.	11:50:38
magic_rb	Say im working on a game, then to nix build it, i need essentially all the annexed files, so the for every build im copying all the assets which can be arbitrarily huge	11:51:13
roberth	That's designing for the current Nix, not the Nix we're promising, fwiw	11:51:33
magic_rb	Even with lazy tree, if im using 40GB, i have to copy 40GB every time no?	11:52:15
magic_rb	Thats how i understood it	11:52:25
roberth	So I guess we have three possible behaviors: deference it completely and return the file contents put it in the store and return store references somehow (more complicated than it seems, but potentially lazy under more circumstances) generate a `.git/annex` which only has the relevant entries (probably annoying to work with, unless your build makes assumptions about `.git/annex`)	11:53:59
roberth	Only if you dereference every annexed file	11:54:31
roberth	To make fetching the annexed files lazy involves: changing fetchTree to not copy everything to the store, returning a path value instead of a store path (or a lazy-trees store path that produces the correct hash part, which has never been done) implementing the annex support in an "event based" manner - not eagerly fetching everything	11:58:34
emily	if the build of a game requires processing every source asset – which I believe is normal – then this does not stop copying every asset anew for every build, even if lazy trees were fully implemented, right?	11:59:20
emily	(maybe you can cut down on it if you can split it into one asset build per derivation since the store paths would remain the same if the individual asset doesn't change?)	11:59:55
emily	(and I guess integration could avoid re-hashing the file to determine that?)	12:00:03
emily	just checking I understand correctly that if all the assets are needed in the same derivation as part of a `src = ./.;` type thing it would still amount to copying the entire thing every time	12:00:24

Show newer messages

Back to Room ListRoom Version: 6