| 20 Jul 2024 |
 Mic92 | In reply to @traxys:familleboyer.net
Hope it was not the 1000 derivation build x) How much memory does it use if you try to build it with nix-fast-build locally? | 20:44:51 |
 Traxys | I tried this command: nix run nixpkgs#nix-eval-jobs -- --workers 1 --check-cache-status --force-recurse --flake 'github:nix-community/nixvim#checks' and that's what used ~4.7 GiB | 20:46:40 |
| Traxys | Well not github though, a local checkout of the PR removing a link farm | 20:47:36 |
| Mic92 | Doesn't sound that crazy | 20:51:12 |
| Mic92 | is your link farm aggregating all plugins in one attribute? | 20:51:25 |
| Traxys | Yeah | 20:51:47 |
| Mic92 | Sounds like something that would actually make parallel evaluation slower | 20:51:50 |
| Traxys | I'd say it had about 300 derivations? | 20:52:12 |
| Traxys | (as a 1000 checks were with all 4 architectures) | 20:52:54 |
| Traxys | We had a link farm I think mostly for the reason fixed by https://github.com/nix-community/buildbot-nix/pull/217
Though I have a question, why does using a link farm consume more RAM than splitting them up? Shouldn't we expect the same result in-fine ? | 20:56:38 |
 magic_rb | Iirc nix doesnt really do GC that well | 20:59:08 |
| magic_rb | So it doesnt GC as much as you would think it should | 20:59:23 |
 aidalgol | Do you mean the nix store, or nix the language and runtime? | 21:38:51 |
| aidalgol | * Do you mean the nix store, or the nix language runtime? | 21:40:34 |
 tomberek | The language GC (to help control memory usage) is quite costly. It is also very conservative, so it is not as beneficial as it might be if we had a dedication allocation and GC system that understood Nix semantics. With additional strictness or linearity analysis it is possible we could collect many more values. Nixpkgs is also a big to blame here, it is built in a way that makes it harder to GC things. | 21:41:35 |
| tomberek | * The language GC (to help control memory usage) is quite costly. It is also very conservative, so it is not as beneficial as it might be if we had a dedication allocation and GC system that understood Nix semantics. With additional strictness or linearity analysis it is possible we could collect many more values. Nixpkgs is also a bit to blame here, it is built in a way that makes it harder to GC things. | 21:41:49 |
 zowoq | In reply to @traxys:familleboyer.net
zowoq I tried running the command you provided with this PR and I get a Maximum Resident Set Size of 4.7 GiB, is this more acceptable? Yes, that will fix it. | 23:42:44 |
| 21 Jul 2024 |
| magic_rb | In reply to @tomberek:matrix.org
The language GC (to help control memory usage) is quite costly. It is also very conservative, so it is not as beneficial as it might be if we had a dedication allocation and GC system that understood Nix semantics. With additional strictness or linearity analysis it is possible we could collect many more values. Nixpkgs is also a big to blame here, it is built in a way that makes it harder to GC things. How does nixpkgs contribute to the problem? | 08:01:54 |
 emily | everything is circular | 08:16:41 |
 @aloisw:kde.org | In reply to @magic_rb:matrix.redalder.org
How does nixpkgs contribute to the problem? Nixpkgs derivations remember too much of their inputs. | 08:29:44 |
| Traxys | zowoq we merged the PR that reduced the memory usage. We'd need to ask all open PRs to rebase on main, and we should be good to re-enable buildbot | 13:22:20 |
 Matt Sturgeon | Was our stable branch affected by the issue?
Do we need to disable buildbot on that branch until the change is backported, or since it is a low traffic branch is it enough that we simply avoid triggering any CI builds on stable for now? | 13:39:16 |
| Traxys | I don't really know :/ | 14:00:57 |
