 | Nixpkgs Stdenv | 221 Members |
| 70 Servers |
| Nixpkgs Stdenv | 221 Members |
| 70 Servers |
| Sender | Message | Time |
|---|---|---|
| 19 Nov 2024 | ||
 sterni | sorry if I've been difficult :) | 18:01:14 |
 emily | eh, it's just a bunch of understandable but colliding priorities I think :) | 18:05:23 |
| emily | execline wants to not worry about its own closure size too obsessively, Nixpkgs tests want to be as cheap as possible to run, cross wants to not depend on execution of the host platform, tests need running, binaries want to dynamically generate things | 18:06:11 |
| emily | it's all an ugly mess but at least the simple decoupling should reduce some of the pressure points there | 18:06:39 |
| emily | I think in an ideal world we would have a cross setup where tests and generation of completions/introspection data/etc. are always in separate derivations that can be opted out of, and can run at the Nix level either through emulation or remotely on a (presumably underpowered for building) native machine | 18:07:50 |
| emily | but that's a massive project | 18:08:07 |
| sterni | also for that we would need to solve the binary cache problem™ since you'd probably need to have intermediate artifacts (post buildPhase) as a generic mechanism | 18:38:41 |
| sterni | which obviously causes somewhat of a space problem | 18:38:51 |
| emily | yes | 18:39:37 |
| emily | I just file every improvement above a certain level of architectural ambition as "TODO in my Nix redesign" | 18:39:57 |
| emily | which allows me to remain ruthlessly pragmatic without feeling like I'm betraying my fundamental values :p | 18:40:17 |
 p14 | emily: I’ve been thinking a bit about what you were saying about separation of interface vs implementation for CA builds. One concern is: if you switch out the implementation, should that not also require a rebuild of downstream software which uses the implementation during the build at least? I was thinking of the openssl case where ideally updating openssl doesn’t require a rebuild of the world, but as soon as you have a package which in checkPhase exercised the implementation of openssl, that package could be broken by the new implementation but would not have its tests run. I guess if you depend on the implementation in order to build (or test) then that would require passing the implementation in; and therefore would condition the build on the implementation. I also wonder if I am talking myself into the concept that testing would have to happen in a separate derivation, essentially, which consumes the implementation. | 19:32:34 |
| emily | it would have its tests run – tests would happen after the relinking stage | 19:33:06 |
| emily | you could avoid these being in serial by using something like Ninja's "validations" | 19:33:17 |
| emily | (with Nix work) | 19:33:21 |
| p14 | But even if you split testing you could still have stuff whose build unfortunately depends on implementation and not just the interface. | 19:33:22 |
| emily | tests don't have to block the build | 19:33:27 |
| emily | as long as they prevent it from being successful if they fail | 19:33:35 |
| p14 | “Ninja’s validations”? | 19:33:46 |
| emily | so tests are much more embarrassingly parallelizable in theory | 19:33:48 |
| emily | uhh, it was discussed in the thread :) | 19:33:53 |
| p14 | Flew over my head if so, apols. | 19:34:13 |
| emily | you can say that build X only succeeds if build Y succeeds, but unlike "X depends on Y", build Z that depends on X can run before Y completes | 19:34:36 |
| emily | https://ninja-build.org/manual.html#validations | 19:34:42 |
| emily | this means that tests could basically happen in parallel to each other and all the relinking | 19:34:55 |
| emily | you don't have to finish the tests for libfoo to test application bar that depends on libfoo | 19:35:05 |
| emily | they can run simultaneously | 19:35:08 |
| emily | (though again this would require support in Nix) | 19:35:11 |
| emily | as far as "But even if you split testing you could still have stuff whose build unfortunately depends on implementation and not just the interface." goes, it would be statically ruled out by the scheme – with only the stubs, things would just not run | 19:35:33 |
| emily | stuff that absolutely needs it would have to have the bits that need runtime support split out (ties in to the cross conversation we just had in here actually), or to depend directly on relinked stuff and thereby be exempt from the scheme | 19:35:57 |