| 1 Jul 2025 |
 roberth | To make fetching the annexed files lazy involves:
- changing fetchTree to not copy everything to the store, returning a path value instead of a store path (or a lazy-trees store path that produces the correct hash part, which has never been done)
- implementing the annex support in an "event based" manner - not eagerly fetching everything
| 11:58:34 |
 emily | if the build of a game requires processing every source asset – which I believe is normal – then this does not stop copying every asset anew for every build, even if lazy trees were fully implemented, right? | 11:59:20 |
| emily | (maybe you can cut down on it if you can split it into one asset build per derivation since the store paths would remain the same if the individual asset doesn't change?) | 11:59:55 |
| emily | (and I guess integration could avoid re-hashing the file to determine that?) | 12:00:03 |
| emily | just checking I understand correctly that if all the assets are needed in the same derivation as part of a src = ./.; type thing it would still amount to copying the entire thing every time | 12:00:24 |
 magic_rb | Yeah this is what im saying. Say youre building a godot game, you end up with src = ./., you cant do much with uh, whats was it filterTree | 12:03:33 |
| roberth | The store references choice does make that easier to achieve. Otherwise, we're looking representations of store objects that don't reside in a real filesystem but in a FUSE store, and an underlying ca-store like tvix/snix. That would be great to have, but it's a lot of work | 12:03:44 |
| emily | I think the practical solution is just to poke holes in the sandbox to access the underlying content-addressed annex store | 12:05:47 |
| emily | and rewrite symlinks to point to it before you run your build | 12:05:56 |
| roberth | Right, so to clarify, if your main goal is to distribute a built game, you should not use builtin fetchers for anything but Nix expressions, because those will be intermediate results, and only in case of FODs it's possible to optimize those away by virtue of the dependent being available and having no reference to the source | 12:05:59 |
| roberth | So what we're left with is the "developer" use case | 12:06:15 |
| emily | it is "reasonably" pure since things are still addressed by hash but it seems to me the only way to get builds involving >40 GiB of assets without years of experimental work on the Nix end | 12:06:25 |
| roberth | For development, presumably you want a quick build, so making asset processing fine grained with a derivation per asset would be the thing to do | 12:06:59 |
| magic_rb | To clarify, i dont have an immediate use case for this. It would be nice, but so far i dont have a game id be developing lol | 12:07:17 |
