| 21 Oct 2025 |
 Sergei Zimmerman (xokdvium) | In reply to @juliankuners:matrix.org
Hmm, considering all the past CI issues, I think I spotted one point in CI logs where the issue starts happening, as it logs the additional unpacking [...] log that later is not printed anymore. Though I cannot confirm whether this is the only place where the issue starts to appear. This code snippet in CI executes nix flake metadata . --refresh. It runs in a checkout without submodules, and I suppose without the empty git submodule directories.
Do you think this could populate the cache with an invalid entry?
Hm, a local checkout might do that yeah. The git fetcher might consider it to be cacheable and doesn’t take into account empty subdirs | 14:27:23 |
 Julian | As I am thinking about ways to work around this, I feel like all workarounds are very hacky. Is this wrong caching behaviour that I observed covered by the above mentioned PR? Or would it be an on-going issue without clear solution, because submodules are involved? Just wondering whether there's a feasible fix in the meantime or whether it's a bigger scale on-going issue. | 14:37:45 |
| Sergei Zimmerman (xokdvium) | In reply to @juliankuners:matrix.org
As I am thinking about ways to work around this, I feel like all workarounds are very hacky. Is this wrong caching behaviour that I observed covered by the above mentioned PR? Or would it be an on-going issue without clear solution, because submodules are involved? Just wondering whether there's a feasible fix in the meantime or whether it's a bigger scale on-going issue. Reasonably we could just avoid caching the fetchToStore entry if submodules are present in the local checkout. A more clever solution would be to filter out the empty subdirs for submodules when fetching to the store from a local checkout. The first solution seems more immediate and backportable. Submodules + caching is definitely an ongoing issue | 14:45:20 |
| Julian | Re-looking at the CI logs, it's probably not immediately the checkout problem, as it the mis-cached repository is another repository and not the ceckout (that is also not a submodule or anything) :/. Sorry for the confusion. | 14:48:26 |
| Julian | For clarification. The repositories have submodules, but none are used for nix flakes and derivations. | 14:49:13 |
| Sergei Zimmerman (xokdvium) | I think I understand what’s going on. Will poke around soonish and see | 15:13:25 |
| Julian | Yea, me somwhat too. It is actually the checkout issue, but it's caused by a checkout of the mis-cached repository in another CI on the same CI machine, see here. The timestamp of the CI job and the cache entry in the sqlite database match exactly, same as the wrong nix store path 72alw9m62226brv4v4m98fqrk31mlp34. | 15:21:07 |
| Julian | I created a respective issue, highlighting the I suppose actual root cause, all things considered https://github.com/NixOS/nix/issues/14317 | 17:03:51 |
 dramforever | In reply to @juliankuners:matrix.org
I created a respective issue, highlighting the I suppose actual root cause, all things considered https://github.com/NixOS/nix/issues/14317 oh yeah that's a duplicate of https://github.com/NixOS/nix/issues/13698 | 17:13:40 |
| dramforever | guess i was a few hours late to the discussion | 17:15:47 |
| Julian | Thank you, I missed this one during my issue search. I closed it and linked to it in the respective other issue. | 17:19:57 |
|  @echobc:matrix.org joined the room. | 18:09:44 |
|  NixOS Moderation Bot banned @echobc:matrix.org (<no reason supplied>). | 18:09:44 |
 niksnut | learned today that using std::string for large buffers is very inefficient (huge kernel overhead): https://github.com/DeterminateSystems/nix-src/pull/238/commits/edf45d6e1158a059e7ded5460d2a3947dc35fdf8 | 20:44:48 |
