| 28 Jun 2025 |
 vcunat | Indeed. On the last restart I looked at the tail of the log periodically, and it really looks like the 2h silent timeout. Always in the same place. | 15:59:19 |
| vcunat | Ah, that was step 7. And the step 8 also looks stuck the same now. | 16:04:33 |
| vcunat | At this point I don't think more retries will help. | 16:04:56 |
 hexa | Receiving outputs | 16:12:41 |
| hexa | 2h48m | 16:12:48 |
| vcunat | 🎉🎉🎉 | 16:21:36 |
| vcunat | Though there's some more general problem around this on 24.11. I see the same with other variants: devedition, beta, thunderbird-latest, librewolf. | 16:35:50 |
 emily | I feel the silent timeout is too short in general | 16:38:13 |
| emily | but maybe you could turn on some spammy logging in the build like we did with LLVM | 16:38:22 |
 aloisw | That won't help when the linker is swapping for over 1 hour because it uses 50 gigabytes of RAM. | 16:38:52 |
| emily | make it print every time it dereferences a pointer :P | 16:40:04 |
| emily | do we link browsers with the GNU linker currently? | 16:40:20 |
| emily | if yes, we should consider… not doing that | 16:40:25 |
| aloisw | All the Geckos I have seen building linked using lld. | 16:40:54 |
| aloisw | It is unclear whether they are supposed to use 50 gigabytes, as https://firefox-source-docs.mozilla.org/setup/linux_build.html states 4 GB should be enough although 8 are recommended. | 16:43:01 |
| emily | I disbelieve 4 GiB. how long ago was that written | 16:44:06 |
| aloisw | I don't doubt it if you have enough swap and time. Although the 30 GB of disk are not enough in this case because even the swap needed is more. | 16:46:39 |
| vcunat | Swap usage is minimal. (all these builds run just on this particular machine)
https://grafana.nixos.org/d/rYdddlPWk/node-exporter-full?orgId=1&var-DS_PROMETHEUS=default&var-job=node&var-node=hopeful-rivest.builder.nixos.org:9100&var-diskdevices=%5Ba-z%5D%2B%7Cnvme%5B0-9%5D%2Bn%5B0-9%5D%2B&from=now-24h&to=now&var-datasource=default&refresh=1m&timezone=browser&viewPanel=panel-78 | 16:48:31 |
| vcunat | A couple worse moments were there, but PSI graph looks still very good to me. | 16:49:42 |
| vcunat | * A couple worse moments were there, but PSI graph looks still very good to me. (very short spikes and just a few percent) | 16:51:46 |
| aloisw | Interesting. There are still some huge mountains that are close to 1 hour that could be Geckos linking but I don't know when the builds were running. | 17:19:32 |
| aloisw | (Of course then the explanation would be that it also uses a lot of CPU independently and the swapping is surprisingly cheap, maybe because it's only swapping out garbage.) | 17:21:16 |
| 29 Jun 2025 |
|  jonhermansen set a profile picture. | 16:58:06 |
| 2 Jul 2025 |
| hexa | when I tried linking firefox on my rock 5b I needed just over 40 GB virtual memory | 04:04:02 |
| hexa | * when I tried linking firefox on my rock 5b I needed just over 40 GB memory | 04:04:07 |
| hexa | most of which was zramswap | 04:04:14 |
| hexa | * most of which came from zramswap | 04:04:19 |
| hexa | so i think it's fair to say that this document is a bit dated | 04:04:32 |
| hexa | especially since linking with lld is not influenced by job count or anything | 04:04:55 |
| aloisw | The thing it could be influenced by is whether debug information is included. No idea how large the impact is though. | 04:14:30 |
