| 25 May 2021 |
 andi- | Something like a no-op profile to figure out if that is reproducible | 23:52:43 |
 baloo | well, there is the non-optimized (the one we now run, which is reproducible), the profiledbootstrap (the one we used to run), and the autoprofiled (perf based, which is deemed non-reproducible) | 23:54:48 |
| andi- | Are we running the PGO part with just one build job at a time? I just came across a random pastebin stating it would get "confused" otherwise.... | 23:55:02 |
| baloo | tried that without luck | 23:55:36 |
| baloo | (tried to inject -j1 in the training stage (the one capturing the feedback)) | 23:56:16 |
| baloo | my plan from this point, is to set up the build on two machines, and run it stage by stage | 23:56:34 |
| baloo | and compare the intermediate result from one machine to the other | 23:56:43 |
| baloo | and finger crossed find the issue | 23:56:53 |
| baloo | but ... that's a lot of work | 23:57:04 |
| andi- | I wonder if the kernel version leaks into the builds. | 23:57:50 |
| baloo | same kernel version | 23:58:00 |
| baloo | (on the one I tried) | 23:58:11 |
| andi- | two machines with the same CPU result in the same output? | 23:58:20 |
| andi- | (might be harder for us to test unless we rent some machines temprorary) | 23:58:33 |
| baloo | I haven't tried | 23:58:33 |
| andi- | * (might be harder for us to test unless we rent some machines temporary) | 23:58:38 |
| baloo | yeah I can rent two machines I guess | 23:58:46 |
| andi- | That might be something grahamc (he/him) could probably help with. Packet boxes might be really nice for this. | 23:59:15 |
| baloo | yeah :D | 23:59:31 |
| 26 May 2021 |
| baloo | the resource is need is time though | 00:00:07 |
| baloo | I need* | 00:00:11 |
| baloo | I have access to packet servers from work | 00:00:43 |
| andi- | Well perhaps make a multi-multi-multi output derivation first that spits out all the phases and then we can think about running it on different machines | 00:00:46 |
| andi- | or simply copy the build root into an output at the end | 00:01:16 |
| baloo | additional challenge: because the intermediate stages are large, gcc optimizes it and rewrites them | 00:01:31 |
| baloo | so you need to step manually, snapshot it, | 00:01:44 |
| baloo | which is ... and extra hurdle | 00:01:51 |
| andi- | Well perhaps a patch to the Makefiles then? | 00:02:07 |
| andi- | Before each stage copy them to $debug/$stage | 00:02:16 |
| baloo | :D | 00:02:15 |
