| 25 May 2025 |
 hexa | * ignore the ones with the ?prefix= set | 16:54:51 |
 emily | https://channels.nixos.org/?prefix=nixos-unstable/ vs https://channels.nixos.org/nixos-unstable | 16:56:34 |
 dramforever | i just realized the "symlinks" are 404 anyway | 17:09:45 |
| dramforever | anyway thanks for the tips | 17:09:53 |
| 26 May 2025 |
|  @entheogenesis:matrix.org left the room. | 03:59:12 |
|  @federicodschonborn:matrix.org changed their display name from This LEGO® Worm™ is licensed under the terms of the he/him or they/them pronouns, at your choice to LEGO® Worm™ (Asbestos Flavored!). | 13:15:48 |
| 27 May 2025 |
| @federicodschonborn:matrix.org changed their display name from LEGO® Worm™ (Asbestos Flavored!) to LEGO® Worm™ (Now Only 1 krad!). | 14:46:06 |
| 28 May 2025 |
 uep | trunk-combined builds seem to be inordinately slow lately. I realise we're in that period where there's an extra release branch in the workload, but even so.. it seems worse than it has been in other cycles. Is it just my perception, or something else contributing as well? | 03:18:44 |
| hexa | we're coming out of a staging-24.11 cycle, not sure | 10:03:52 |
| hexa | we use grafana to observe the pace of hydra usually | 10:04:21 |
| hexa | and what you can see is that is has no linux runnables | 10:04:51 |
 Vladimír Čunát | The "builds ingestion" is almost always what blocks us (if speaking about linux). | 10:05:56 |
| Vladimír Čunát | That's a long-term property. | 10:06:28 |
| Vladimír Čunát | "Steps completed per minute" isn't much lower than usually, I'd say. | 10:07:16 |
| uep | i'm going by https://hydra.nixos.org/jobset/nixos/trunk-combined, and the fact that for the last.. quite a few days, it seems the build from the previous day hasn't finished by the time the next one starts. And even with relatively low numbers of jobs to run (8k, vs 120k) it's taking more than a day to finish.
I realise this isn't the total picture, i'm not looking at overall throughput or jobs | 10:11:42 |
| hexa | this relates to the "build ingestion" in that we don't currently get enough runnables to saturate our linux builders | 10:14:30 |
| hexa | usually restarting the queue-runner can tip the balance again | 10:15:03 |
| Vladimír Čunát | Redacted or Malformed Event | 10:15:16 |
| uep | oh, build ingestion is on-boarding of jobs to run, not ingestion of build results... makes more sense now | 10:24:33 |
| hexa | https://github.com/NixOS/hydra/commit/720db63d52ebcbda617603e7aa5b5c750cc6afec was the last change to the scheduling logic | 10:29:52 |
| hexa | we've run with that since around april 29th I would say | 10:31:51 |
| hexa | but that's in the dispatcher, not the ingestor | 10:32:15 |
| hexa | * but that's in the dispatcher, not the queue-monitor | 10:32:41 |
| hexa | and ingestion business is determined here https://github.com/NixOS/hydra/blob/master/src/hydra-queue-runner/queue-monitor.cc#L54-L82 | 10:33:22 |
| hexa | * and ingestion busyness is determined here https://github.com/NixOS/hydra/blob/master/src/hydra-queue-runner/queue-monitor.cc#L54-L82 | 10:33:34 |
| uep | i dearly love postres notify, but the fact that it's bundled into a transaction sometimes really hurts | 10:36:45 |
| uep | * i dearly love postgres notify, but the fact that it's bundled into a transaction sometimes really hurts | 10:37:04 |
 K900 | The entire queue setup is one huge crime against postgres | 10:37:29 |
| K900 | Unfortunately | 10:37:47 |
| Vladimír Čunát | It often feels like restarting the queue runner helps. I tried that again during this discussion, and it does seem like getting through trunk-combined faster (including the Steps completed per minute graph). No idea why this happens, and yes - during the restart we lose already ingested builds that haven't finished, and startup takes some time, so it's not always a win. | 11:02:13 |
