| 12 Feb 2024 |
 cole-h | ofborg does not use 32 cores for 32 evaluations 😆
It only has enough memory to run 3 evaluations on one machine most times. (256GiB total, each eval-outpaths takes ~60+GiB of memory.) | 15:03:16 |
| cole-h | So, provided there's no huge build taking place on the same box, the other cores are usually sitting around twiddling their thumbs. | 15:04:13 |
 raitobezarius | In reply to @infinisil:matrix.org
Because if it's already using 32 cores for 32 separate evaluations, that PR would not help at all (in fact it would make it a lot worse, because each parallel unit of work duplicates work..) well it's not like Nix is smart and knows how to deal with values appropriately | 15:51:22 |
| raitobezarius | so not sure if that would be that wasteful | 15:51:26 |
| raitobezarius | measurements are important here | 15:51:28 |
| raitobezarius | because the GC is really in a bad shape | 15:51:36 |
 @infinisil:matrix.org | I see, so it sounds like it would still help a bunch | 16:29:41 |
 trofi | It would improve in a sense that that PR does not really list a superset of all hydra attributes. For example it skips all recursive attributes like pythonPackages or haskellPackages. | 21:42:19 |
| trofi | But I agree nix's RAM usage for tree traversals is unusable on large trees. https://github.com/NixOS/nix/issues/9671 feels like being a bit more eager about evicting unreferenced values would help a lot. nix-env -qa is a bit trickier as package list is built and sorted outside nix expressions. | 21:44:08 |
| 14 Feb 2024 |
 teto | How can I see the actual ofborg queue ? I would like to see where my PR (288761) darwin job is to get an idea how long it will take. I see there are 133 jobs | 19:35:26 |
 7c6f434c | Based on another PR and the stats, I think the two Darwin queues are comparable, so one of them reaching that PR is a good sign… | 19:40:14 |
| cole-h | You can't, sorry. | 19:40:31 |
| teto | I suspected so :'( ty | 19:59:35 |
| 16 Feb 2024 |
 @a-n-n-a-l-e-e:matrix.org | not sure if this is the right channel but: i think i'm seeing some failures on ofborg aarch64-darwin that aren't on hydra. hydra builds are passing but ofborg builds are failing. i created ofborg builds by creating a PR and adding a comment in postPatch to trigger a rebuild but seem to be failing 100% while running unit tests. I noticed issues when trying to upgrade the same package and which is why i backgracked to a known good build (green hydra) but am still seeing failures. | 09:11:01 |
| @a-n-n-a-l-e-e:matrix.org | https://hydra.nixos.org/job/nixpkgs/trunk/python311Packages.duckdb.aarch64-darwin/all | 09:11:22 |
| @a-n-n-a-l-e-e:matrix.org | failures: https://logs.ofborg.org/?key=nixos/nixpkgs.289199&attempt_id=1451caa6-78d6-4b52-80b1-abcbaaa29298 (all aarch64-darwins are fails) from PR https://github.com/NixOS/nixpkgs/pull/289199/checks?check_run_id=21646004423 | 09:12:26 |
| 17 Feb 2024 |
| @a-n-n-a-l-e-e:matrix.org | race in unit test -- thought i disabled it but did not. race not triggered on hydra as i guess hydra is machines are slower. | 06:14:16 |
| 20 Feb 2024 |
 @janik0:matrix.org | did the ofborg storage run full some time ago? https://github.com/NixOS/nixpkgs/pull/276376 | 16:59:07 |
| cole-h | It did last month, yes, but that should be resolved now. | 17:00:15 |
 Lily Foster | In reply to @cole-h:matrix.org
It did last month, yes, but that should be resolved now. nix should be automatically garbage collecting with the min-free option, right? i forget if we had something like that set on the ofborg builders | 17:01:08 |
| cole-h | It's not actually space that's the issue, but inodes, since they run on ext4 | 17:01:26 |
| Lily Foster | ah | 17:01:33 |
| Lily Foster | well. that would do it | 17:01:36 |
| Lily Foster | (i wonder if auto-optimise-store would be reasonable to save on inodes 🤔) | 17:02:08 |
| Lily Foster | (idk maybe they're already doing that too) | 17:02:13 |
| cole-h | They used to run on zfs, but when EM refreshed their hardware, we would sometimes get machines that didn't support EFI (this was like 2+ years ago now so I don't remember the exact details, but something something no EFI = no ZFS booting) | 17:02:29 |
| @janik0:matrix.org | okay, thank's that explains the error :) | 17:03:59 |
| 22 Feb 2024 |
 hexa | can we? https://github.com/samueldr/ofborg-viewer/pull/28 | 00:13:11 |
| hexa | please? | 00:13:14 |
| hexa | dark mode! | 00:13:19 |
