10 Apr 2023 |
hexa | In reply to @cole-h:matrix.org Try now? works | 13:46:35 |
hexa | thanks | 13:46:40 |
7c6f434c | I would say that 5000 packages from many of the lighter ecosystems are still lighter than a single Chromium rebuild. | 13:47:43 |
K900 | For Hydra, absolutely not | 13:48:29 |
K900 | Chromium gets thrown on a very wide machines | 13:48:38 |
K900 | All the other packages get like two cores | 13:48:45 |
K900 | And the coordinator overhead is unfortunately massive | 13:48:56 |
7c6f434c | Well, about wide machine, I wonder if I can find 5000 packages that are still lighter than Gimp, or where the cutoff for getting such a ton of cores lies | 13:50:32 |
7c6f434c | Although coordinator overhead can always be high enough to overshadow the builds, maybe | 13:51:08 |
K900 | Chromium is 2 hours on a big-parallel machine: https://hydra.nixos.org/build/215069962#tabs-buildsteps | 13:51:45 |
K900 | That's probably comparable to the time it would take Hydra to schedule 5000 no-ops | 13:52:37 |
7c6f434c | A few thousands Lisp packages were like an hour on my laptop, I think… | 13:52:44 |
K900 | Which is admittedly not a good look for Hydra | 13:52:46 |
K900 | But what can you do | 13:52:48 |
K900 | In reply to@7c6f434c:nitro.chat A few thousands Lisp packages were like an hour on my laptop, I think… Yes, this is specifically a Hydra problem to a certain extent | 13:53:15 |
K900 | The cutoff number could be way higher if Hydra could scale better | 13:53:32 |
K900 | As we have a lot more compute capacity hypothetically available than we actually use | 13:53:50 |
7c6f434c | Speaking of what we can do… this is specifically Hydra scheduler, right? So bundling lower-impact light-weight packages as dependencies of a single named Hydra job would help? | 13:58:02 |
K900 | No | 13:58:08 |
K900 | Hydra schedules things per derivation | 13:58:13 |
K900 | Not per Hydra job | 13:58:20 |
cole-h | (Maybe this conversation should be moved elsewhere? :) | 13:58:26 |
K900 | #hydra:nixos.org probably | 13:58:35 |
K900 | Or #infra:nixos.org | 13:58:45 |
tea | In reply to @cole-h:matrix.org I'm not sure it ever did...? AFAIK, it only ever built packages based on the commit message in a given PR. yall were right | 14:02:13 |
tea | it was nixpkgs-update: | 14:02:18 |
tea | https://discourse.nixos.org/t/nixpkgs-update-runs-nixpkgs-review-on-every-pr/6701 | 14:02:22 |
tea | Redacted or Malformed Event | 14:02:54 |
13 Apr 2023 |
K900 | It happened again. | 08:45:32 |
K900 | https://github.com/NixOS/nixpkgs/pull/215109 | 08:45:33 |