| 19 Jul 2024 |
 magic_rb | Yeah that should be fine, ill test it later today | 16:59:34 |
 Mic92 | In reply to @mattsturg:matrix.org
I think my 2018 laptop uses 100% all cores for most of the test build, my desktop is less affected.
We used to use around 32GB RAM, but Traxys' recent change dropped this to around 3GB (which fixed some of our other buildbot issues, as mentioned on #227) We should also soon be better at handling many derivations in .checks: https://github.com/nix-community/buildbot-nix/pull/217 | 18:03:44 |
 Traxys | Yup I'm subscribed to that issue :D I think it could help a lot, as we may be able to ditch our link farm :D | 18:05:33 |
| Traxys | * Yup I'm subscribed to that PR :D I think it could help a lot, as we may be able to ditch our link farm :D | 18:05:39 |
| Traxys | Ok so we have on the order of 1100 builbot jobs :x Would it risk to overwhelm something in builbot? | 18:35:47 |
| Traxys | * Ok so we have on the order of 1100 builbot jobs :x Would it risk to overwhelm something in builbot? (if we ditch the link farm) | 18:36:10 |
| magic_rb | In reply to @traxys:familleboyer.net
Ok so we have on the order of 1100 builbot jobs :x Would it risk to overwhelm something in builbot? (if we ditch the link farm) do it, i dont know, itll create lots of builds for sure, but to the github side it wont be a problem thats taken care of, by that open PR, so after, test it and we'll see :) | 20:34:38 |
| Traxys | I have an WIP PR doing that: https://github.com/nix-community/nixvim/pull/1898
What seems to happen is that builds are a bit long, mostly on waiting for builders as each derivation is very quick to compute (https://buildbot.nix-community.org/#/builders/43/builds/294) | 20:41:17 |
| Traxys | Though with more than a 1000 checks it's a bit of a mess on the github side, but that should be fixed by the buildbot PR :D | 20:41:48 |
| magic_rb | Okay im genuinely impressed this works 👀 | 20:49:40 |
| Traxys | Oops I think something broke buildbot, I get connection refused while trying to evaluate a flake (https://buildbot.nix-community.org/#/builders/43/builds/307) | 22:31:45 |
| Traxys | Hope it was not the 1000 derivation build x) | 22:32:01 |
 Gaétan Lepage | Me too | 22:36:45 |
| Gaétan Lepage | 
Download clipboard.png | 22:37:09 |
 Matt Sturgeon | We've had those kinda errors randomly in the past, but the timing of it happening after testing #1898 doesn't look good... | 22:51:13 |
| 20 Jul 2024 |
 zowoq | I've restarted it. | 00:24:58 |
|  @brisingr05:matrix.org left the room. | 05:48:41 |
| Traxys | zowoq I tried running the command you provided with this PR and I get a Maximum Resident Set Size of 4.7 GiB, is this more acceptable? | 13:36:30 |
| Mic92 | Nscloud runners also support a /nix cache across CI runs: https://github.com/nix-community/nix-eval-jobs/pull/318/files | 19:56:05 |
| Mic92 | In reply to @traxys:familleboyer.net
Hope it was not the 1000 derivation build x) How much memory does it use if you try to build it with nix-fast-build locally? | 20:44:51 |
| Traxys | I tried this command: nix run nixpkgs#nix-eval-jobs -- --workers 1 --check-cache-status --force-recurse --flake 'github:nix-community/nixvim#checks' and that's what used ~4.7 GiB | 20:46:40 |
| Traxys | Well not github though, a local checkout of the PR removing a link farm | 20:47:36 |
| Mic92 | Doesn't sound that crazy | 20:51:12 |
| Mic92 | is your link farm aggregating all plugins in one attribute? | 20:51:25 |
| Traxys | Yeah | 20:51:47 |
| Mic92 | Sounds like something that would actually make parallel evaluation slower | 20:51:50 |
| Traxys | I'd say it had about 300 derivations? | 20:52:12 |
| Traxys | (as a 1000 checks were with all 4 architectures) | 20:52:54 |
| Traxys | We had a link farm I think mostly for the reason fixed by https://github.com/nix-community/buildbot-nix/pull/217
Though I have a question, why does using a link farm consume more RAM than splitting them up? Shouldn't we expect the same result in-fine ? | 20:56:38 |
| magic_rb | Iirc nix doesnt really do GC that well | 20:59:08 |
