| 21 Oct 2025 |
 Julian | For clarification. The repositories have submodules, but none are used for nix flakes and derivations. | 14:49:13 |
 Sergei Zimmerman (xokdvium) | I think I understand what’s going on. Will poke around soonish and see | 15:13:25 |
| Julian | Yea, me somwhat too. It is actually the checkout issue, but it's caused by a checkout of the mis-cached repository in another CI on the same CI machine, see here. The timestamp of the CI job and the cache entry in the sqlite database match exactly, same as the wrong nix store path 72alw9m62226brv4v4m98fqrk31mlp34. | 15:21:07 |
| Julian | I created a respective issue, highlighting the I suppose actual root cause, all things considered https://github.com/NixOS/nix/issues/14317 | 17:03:51 |
 dramforever | In reply to @juliankuners:matrix.org
I created a respective issue, highlighting the I suppose actual root cause, all things considered https://github.com/NixOS/nix/issues/14317 oh yeah that's a duplicate of https://github.com/NixOS/nix/issues/13698 | 17:13:40 |
| dramforever | guess i was a few hours late to the discussion | 17:15:47 |
| Julian | Thank you, I missed this one during my issue search. I closed it and linked to it in the respective other issue. | 17:19:57 |
|  @echobc:matrix.org joined the room. | 18:09:44 |
|  mjolnir banned @echobc:matrix.org (<no reason supplied>). | 18:09:44 |
 niksnut | learned today that using std::string for large buffers is very inefficient (huge kernel overhead): https://github.com/DeterminateSystems/nix-src/pull/238/commits/edf45d6e1158a059e7ded5460d2a3947dc35fdf8 | 20:44:48 |
| Sergei Zimmerman (xokdvium) | This is sort of more about doing unnecessary construction/destruction of an object. Not special to std::string in any way | 20:55:40 |
| niksnut | it's a result of having a large contiguous allocation (so it would also affect std::vector<char>) | 20:56:21 |
| Sergei Zimmerman (xokdvium) | Yeah, it's much better to allocate once and reuse that allocation as you did there | 20:56:45 |
| niksnut | a data type that consists of a vector of buffers would avoid this problem | 20:57:06 |
| Sergei Zimmerman (xokdvium) | I did a similar cleanup in filetransfer..cc at some point | 20:57:07 |
| Sergei Zimmerman (xokdvium) | You'd still benefit from allocating it only once though | 20:57:23 |
| Sergei Zimmerman (xokdvium) | Going through malloc/free is still a function call through PLT and doing that in a loop is kind of expensive anyway | 20:58:05 |
| Sergei Zimmerman (xokdvium) | https://llvm.org/docs/ProgrammersManual.html#vector | 20:58:58 |
| niksnut | it's funny though that the optimization that skipped parsing 15 GB of NARs (https://github.com/DeterminateSystems/nix-src/pull/238/commits/1f8d587a0df8f9de366640831dade43d17021c30) had basically no observable effect | 21:01:55 |
| niksnut | it's completely dwarfed by the memory allocation / page fault overhead | 21:02:07 |
| Sergei Zimmerman (xokdvium) | Yeah that will do it certainly. Also having a too large buffer on the stack is bad:
https://github.com/NixOS/nix/pull/13877 | 21:03:27 |
| Sergei Zimmerman (xokdvium) | The stack pointer does get decremented one page at a time. Is that the default behavior or some hardening flag? | 21:04:10 |
| niksnut | surprising since stack pages should stay around once paged in | 21:07:22 |
| niksnut | though there is some overhead to handle guard pages | 21:07:42 |
| niksnut | so it has to touch at least 1 byte every 4096 bytes | 21:07:52 |
| Sergei Zimmerman (xokdvium) | Yeah that was the overhead. A loop over all the pages | 21:08:28 |
| Sergei Zimmerman (xokdvium) | 1.23 │ lea -0x10000(%rsp),%r11
0.23 │ 15: sub $0x1000,%rsp
1.01 │ orq $0x0,(%rsp)
59.12 │ cmp %r11,%rsp
0.27 │ ↑ jne 15
| 21:08:42 |
| niksnut | right, that's to avoid a segfault if you have guard pages enabled (which I think is the default) | 21:09:13 |
| niksnut | I would expect the overhead for that loop to be pretty trivial though | 21:09:31 |
| niksnut | in the case where the pages are present | 21:09:52 |
