| 25 Nov 2024 |
 emily | (this came up in the context of a long discussion about dynamic library stubs / relinking to avoid mass rebuilds from ABI-compatible library changes, https://discourse.nixos.org/t/why-does-the-nixos-infrastructure-have-to-be-hosted-in-a-centralized-way/46789/32 onwards but it's quite long) | 19:31:49 |
| emily | anyway, +1 to everything that gets rid of the phantom dependencies at the Nix level | 19:32:04 |
|  @apurba007:matrix.org left the room. | 19:34:49 |
 p14 | emily: I did a small poc of this over the weekend. And looked into the effects. It looks like a plausible thing to me. Interesting question of how/whether it can be wired into nixpkgs in general without it requiring a big-bang rewrite. | 19:48:30 |
| p14 | It is very cool watching a dependency rebuild but seeing an instant relink for an expensive to build dependent. | 19:49:33 |
| emily | nice | 19:50:26 |
| emily | would be cool to see your prototype | 19:50:29 |
| p14 | Afk right now, will get back to you on that. | 19:51:19 |
 John Ericson | @emilazy:matrix.org I like your thinking in that thread. I agree the interesting thing is not perfect minimal rebuilds, but separating concerns purely at scale | 19:58:17 |
| emily | my ideal is to get away from staging entirely and do everything as a bors-style merge queue where every single change leaves everything green, but even applying all the ideas in that thread to the maximum extent probably still leaves a very substantial gap away from making that viable :) | 19:59:50 |
| emily | but it's frustrating that we at least have a way to deploy one-byte security patches to libraries at scale that doesn't involve compromising the model like replaceDependencies but – we don't do it | 20:00:19 |
| emily | (for understandable reasons, given Nixpkgs legacy, but still.) | 20:00:29 |
| John Ericson | If we make sure we don't starve the remaining mass rebuilds, I think we can do it! | 20:00:41 |
| p14 | That would be super cool. | 20:01:26 |
| emily | it's a shame that Unix ended up with this model where you have the runtime executable code and the linker ABI information in the same file, even though the compilation stage just doesn't care about the former at all | 20:01:54 |
| emily | the LLVM stub stuff is cool but it still essentially involves making hack ELF files to get tools to cooperate | 20:02:10 |
| p14 | John Ericson: i am excited to see work being thought about in the CA world. Really want to see this happen. | 20:02:21 |
| emily | macOS is ahead here – all the system libraries ship as YAML stub files that just list symbols in the official SDKs | 20:02:26 |
 trofi | In reply to @Ericson2314:matrix.org
I know finally understand what trofi was doing long ago in mkDerivation, and I agree that's the right thing to do for now Restored as https://github.com/NixOS/nixpkgs/pull/359098 :) | 20:14:41 |
| John Ericson | trofi: I think it can go directly to master, no? | 20:21:58 |
| John Ericson | not an official mass rebuild :) | 20:22:02 |
| p14 | In reply to @emilazy:matrix.org
would be cool to see your prototype https://gist.github.com/pwaller/31501f4068dfd0f94d867a1aa2cc0be8 | 20:36:01 |
| p14 | Annoyingly, can't put a directory in a gist, so settled on txtar instead. | 20:36:42 |
| p14 | The next trick will be to write this as an overlay, stdenvStage, or minimal patch against nixpkgs that does this to everything through arcane magic. | 20:37:25 |
| emily | I think you can – just add multiple files and it can be git cloned? | 20:37:34 |
| emily | we need some of this ANSI art in Nixpkgs | 20:37:39 |
| p14 | :D | 20:37:44 |
| p14 | You can put multiple files in, but not multiple directories. They get rejected by the git remote when you try to push them. | 20:37:59 |
| p14 | It was a fun little exercise but I can see there is quite a bit more work involved even when it comes to handling a single package. I'm actually not sure what to reach for to do something as simple as replacing nix paths in a binary; patchelf obviously is the thing to reach for when the strings appear in a specific place, but substituteAll barf'd on the null characters. | 20:39:53 |
| emily | I think Nixpkgs just uses sed for that | 20:40:30 |
