| 15 May 2024 |
hexa | reason is that the service sits at 45 GB of memory usage and evals are getting oom-killed | 10:06:04 |
@lotte:chir.rs | In reply to @hexa:lossy.network is there a requirement for hydra-notify to be up for evaluating/scheduling/shipping builds? hydra-notify can be down but stuff like runcommand hooks, github status checks, and log compression will not work | 10:13:19 |
@lotte:chir.rs | in my experience | 10:13:24 |
@lotte:chir.rs | hydra-evaluator schedules evaluations, and hydra-queue-runner schedules builds | 10:14:41 |
@lotte:chir.rs | In reply to @lotte:chir.rs hydra-notify can be down but stuff like runcommand hooks, github status checks, and log compression will not work of course the runcommand hook is probably where you're shipping the build, unless you use an s3 bucket as a backing store for hydra | 10:15:19 |
hexa | it's s3 | 10:15:33 |
hexa | so log compression is probably the most interesting bit | 10:15:40 |
hexa | but also that thing is plugin based, and I don't think we have enabled any | 10:15:52 |
hexa | https://github.com/NixOS/infra/blob/master/delft/hydra.nix#L30-L32 🤷 | 10:17:41 |
| 16 May 2024 |
vcunat | Oh, hydra allows to build the same build twice in parallel on different machines (same hash, dependency of two different jobs). | 07:49:53 |
vcunat | I'm not 100% sure about the overlapping timing, but it really did look that way now. | 07:51:12 |
K900 | Same output hash or same drv hash? | 07:52:17 |
K900 | I think it schedules by drv hash | 07:52:21 |
K900 | So two different FODs with the same output hash can happen | 07:52:33 |
vcunat | Both. It wasn't a FOD. | 07:53:22 |
K900 | Oh no | 07:53:53 |
K900 | Is the toposort broken | 07:53:55 |
vcunat | I retried and it does look like it, at least from web UI. | 07:54:32 |
vcunat | It's a step that doesn't have its own job. | 07:55:01 |
vcunat | Off the top of my head I don't expect real issues from this, mostly just some waste of resources sometimes. | 07:56:07 |
@aloisw:kde.org | Possibly https://github.com/NixOS/nix/issues/7756? | 08:13:52 |
@aloisw:kde.org | * Possibly https://github.com/NixOS/nix/issues/7756 ? | 08:14:21 |
@aloisw:kde.org | * Possibly https://github.com/NixOS/nix/issues/7756 ? | 08:14:29 |
vcunat | I expect it could cause inconsistencies among the non-deterministic stuff (including the log). | 08:34:38 |
| 17 May 2024 |
| @antifuchs:asf.computer left the room. | 13:26:29 |
| @cjbayliss:matrix.org left the room. | 23:47:45 |
| 18 May 2024 |
John Ericson | If anyone wants to rewrite the build system in Meson, I will take it | 00:13:33 |
John Ericson | I am more fond of the Nix/GHC-style fancy single make invocation than automake's $(MAKE) everywhere | 00:14:01 |
John Ericson | I don't think the current Hydra build system has redeeming qualities | 00:14:15 |
| @arcayr:rascals.net left the room. | 08:14:17 |