| 16 Mar 2024 |
 ryantm | closures of inputDerivations https://gist.github.com/ryantm/e7fde94d7f0aeb7ebd334015b310b659 | 04:53:53 |
| ryantm | I don't really know how to run nixos-render-docs will try to learn though. | 04:57:53 |
| ryantm | Found the docs on it in the manual :) | 04:59:49 |
| ryantm | After building and adding one newline to manual.md
[ryantm@replit1:~/p/nixpkgs]$ /run/current-system/sw/bin/time nix-build nixos/release.nix -A manual.x86_64-linux
these 3 derivations will be built:
/nix/store/rdr3b9kqd5qd9rzxqiid68i3znilg95r-lazy-options.json.drv
/nix/store/s2zkfllzrn4c5610irds9305jwhkn8v9-options.json.drv
/nix/store/3cjzvv9gfiwvy91gx1awm06771jcbd02-nixos-manual-html.drv
building '/nix/store/rdr3b9kqd5qd9rzxqiid68i3znilg95r-lazy-options.json.drv'...
building '/nix/store/s2zkfllzrn4c5610irds9305jwhkn8v9-options.json.drv'...
building '/nix/store/3cjzvv9gfiwvy91gx1awm06771jcbd02-nixos-manual-html.drv'...
substituteStream(): WARNING: '--replace' is deprecated, use --replace-{fail,warn,quiet}. (file './manual.md')
/nix/store/jpq9yb4lq14war2r2dq8ymigwa6pgdly-nixos-manual-html
1.56user 0.40system 0:08.52elapsed 23%CPU (0avgtext+0avgdata 444068maxresident)k
8inputs+0outputs (0major+107428minor)pagefaults 0swaps
| 05:01:09 |
| ryantm | Looks like manual.x86_64-linux does not support the inputDerivation passthru param. | 05:04:07 |
| ryantm | * Looks like manual.x86_64-linux does not support the inputDerivation passthru attribute. | 05:04:30 |
 infinisil | In reply to @ryantm:matrix.org
After building and adding one newline to manual.md
[ryantm@replit1:~/p/nixpkgs]$ /run/current-system/sw/bin/time nix-build nixos/release.nix -A manual.x86_64-linux
these 3 derivations will be built:
/nix/store/rdr3b9kqd5qd9rzxqiid68i3znilg95r-lazy-options.json.drv
/nix/store/s2zkfllzrn4c5610irds9305jwhkn8v9-options.json.drv
/nix/store/3cjzvv9gfiwvy91gx1awm06771jcbd02-nixos-manual-html.drv
building '/nix/store/rdr3b9kqd5qd9rzxqiid68i3znilg95r-lazy-options.json.drv'...
building '/nix/store/s2zkfllzrn4c5610irds9305jwhkn8v9-options.json.drv'...
building '/nix/store/3cjzvv9gfiwvy91gx1awm06771jcbd02-nixos-manual-html.drv'...
substituteStream(): WARNING: '--replace' is deprecated, use --replace-{fail,warn,quiet}. (file './manual.md')
/nix/store/jpq9yb4lq14war2r2dq8ymigwa6pgdly-nixos-manual-html
1.56user 0.40system 0:08.52elapsed 23%CPU (0avgtext+0avgdata 444068maxresident)k
8inputs+0outputs (0major+107428minor)pagefaults 0swaps
You can just run a --rebuild or --check. The lazy options thing isn't relevant for nrd | 05:04:37 |
| infinisil | (and it shouldn't be rebuilt anyways, seems like a bug) | 05:04:53 |
| ryantm | [ryantm@replit1:~/p/nixpkgs]$ /run/current-system/sw/bin/time nix-build --check nixos/release.nix -A manual.x86_64-linux
checking outputs of '/nix/store/3cjzvv9gfiwvy91gx1awm06771jcbd02-nixos-manual-html.drv'...
substituteStream(): WARNING: '--replace' is deprecated, use --replace-{fail,warn,quiet}. (file './manual.md')
/nix/store/jpq9yb4lq14war2r2dq8ymigwa6pgdly-nixos-manual-html
1.53user 0.39system 0:04.59elapsed 42%CPU (0avgtext+0avgdata 443704maxresident)k
0inputs+0outputs (0major+107424minor)pagefaults 0swaps
| 05:06:27 |
| infinisil | This works: nix-build nixos --arg configuration '{}' -A config.system.build.manual.manualHTML.inputDerivation | 05:06:38 |
| ryantm | Added nrd input closure to the gist too | 05:09:57 |
| infinisil | ryantm: Oh and nix-build pkgs/top-level/release.nix -A manual is for the Nixpkgs manual | 05:12:08 |
| ryantm | That one would be a more fair comparison since I render the entire nixpkgs manual (also with a search index) | 05:13:21 |
| ryantm | [ryantm@replit1:~/p/nixpkgs]$ /run/current-system/sw/bin/time nix-build --check pkgs/top-level/release.nix -A manual
trace: warning: lib.zipWithNames is a deprecated alias of lib.zipAttrsWithNames.
checking outputs of '/nix/store/3jsvqqgpj2qfbkj2y82w4ac4nnf2g5px-nixpkgs-manual.drv'...
Running phase: unpackPhase
unpacking source archive /nix/store/sn18ydylvr7jsxbhj2yndh9y2s3z55mv-doc
source root is doc
Running phase: patchPhase
Running phase: updateAutotoolsGnuConfigScriptsPhase
Running phase: configurePhase
no configure script, doing nothing
Running phase: buildPhase
substituteStream(): WARNING: '--replace' is deprecated, use --replace-{fail,warn,quiet}. (file './manual.md.in')
Running phase: installPhase
Running phase: fixupPhase
shrinking RPATHs of ELF executables and libraries in /nix/store/m94bxwbilm1hh0dzk53qs7zvdr1vwsps-nixpkgs-manual
checking for references to /build/ in /nix/store/m94bxwbilm1hh0dzk53qs7zvdr1vwsps-nixpkgs-manual...
patching script interpreter paths in /nix/store/m94bxwbilm1hh0dzk53qs7zvdr1vwsps-nixpkgs-manual
/nix/store/m94bxwbilm1hh0dzk53qs7zvdr1vwsps-nixpkgs-manual
0.60user 0.18system 0:02.19elapsed 36%CPU (0avgtext+0avgdata 174568maxresident)k
0inputs+0outputs (0major+39903minor)pagefaults 0swaps
| 05:14:26 |
| ryantm | Very similar times! | 05:15:03 |
| ryantm | Good on you making it that fast with Python :) | 05:16:48 |
| ryantm | Looks like it builds an epub and a single page manual. mmdoc is building a multipage, singlepage, man, epub, and search index, but yeah, maybe pretty close overall speed wise. | 05:18:04 |
| infinisil | Yeah and closure sizes are also comparable (70MB vs 200MB) | 05:22:36 |
| ryantm | Oh wait, I don't think that time is apples to apples:
[ryantm@replit1:~/p/nixpkgs]$ /run/current-system/sw/bin/time nix-build --check pkgs/top-level/release.nix -A manual.inputDerivation
trace: warning: lib.zipWithNames is a deprecated alias of lib.zipAttrsWithNames.
this derivation will be built:
/nix/store/j622rlm0k013mmspbzgkqgd4zl12vvvd-nixpkgs-manual.drv
error: some outputs of '/nix/store/j622rlm0k013mmspbzgkqgd4zl12vvvd-nixpkgs-manual.drv' are not valid, so checking is not possible
Command exited with non-zero status 1
0.59user 0.18system 0:01.53elapsed 50%CPU (0avgtext+0avgdata 174980maxresident)k
0inputs+0outputs (0major+39908minor)pagefaults 0swaps
[ryantm@replit1:~/p/nixpkgs]$ nix path-info --recursive --closure-size --human-readable ./result
/nix/store/m94bxwbilm1hh0dzk53qs7zvdr1vwsps-nixpkgs-manual 2.0M
| 05:22:56 |
| ryantm | The manual is already built as the inputDerivation to that derivation. | 05:23:37 |
| ryantm | Nix is so hard to benchmark with the all the caching and indirection. | 05:24:40 |
| infinisil | ryantm: Not sure what you mean, I'm pretty sure nix-build --check pkgs/top-level/release.nix -A manual is the actual build, it's not just a dependency | 05:25:23 |
| infinisil | And the command you pasted failed | 05:25:33 |
| ryantm | Ah yeah, oops. Okay. | 05:25:53 |
| infinisil | (gotta build without --check first.. yeah not a great interface) | 05:25:58 |
| ryantm | Redacted or Malformed Event | 05:26:59 |
| ryantm | Updated the gist again. Looks like the nixpkgs closure is bigger, but the nixpkgs closure matters less. | 05:28:28 |
| infinisil | Could you post the results in the github issue? I'm overall not very convinced that this is worth it though 😅 | 05:29:31 |
| infinisil | * Could you post the results in the github issue (ideally a nice markdown table with the numbers)? I'm overall not very convinced that this is worth it though 😅 | 05:29:46 |
| ryantm | Yeah there's a lot of hate for C. | 05:30:00 |
