!sBfrWMVsLoSyFTCkNv:nixos.org

OfBorg

170 Members
Number of builds and evals in queue: <TBD>64 Servers

Load older messages


SenderMessageTime
9 Jun 2025
@sigmasquadron:matrix.orgSigmaSquadron joined the room.13:15:34
11 Jun 2025
@7c6f434c:nitro.chat7c6f434c

Hello.

Is there a temporary operational issue with OfBorg currently?

https://github.com/NixOS/nixpkgs/pull/415084/checks?check_run_id=43868776791

copying path '/nix/store/<whatever>' from 'http://build01.ofborg.org'...
error: Nix daemon disconnected unexpectedly (maybe it crashed?)

(Happened to multiple requests on this PR in the last few hours)

08:00:31
12 Jun 2025
@simon.hauser:helsinki-systems.deSimon HauserThere was no "temporary operational issue" i am aware of. I am not sure why this issue happens, so've decided maybe a reboot and 25.05 helps "fix" that issue. I've upgraded the arm builder to 25.05 last evening and i upgraded all x64 builders an hour ago and rebooted all hosts. Maybe it helps to have a newer nix version otherwise ping me again and i'll investigate further. Still missing is 25.05 core01 (rabbitmq, and other services) and darwin builders.07:06:15
@7c6f434c:nitro.chat7c6f434cThanks! Is monitoring (at least the queue size link in the room topic) a part of core01 other services? (Wanted to estimate when to check back on a small-ish build requested a couple hours ago)10:18:14
@simon.hauser:helsinki-systems.deSimon Hauser

no we actually just want to ingest the metrics into the prometheus that is maintained by the nixos infra team, thats still a todo for Janne Heß and myself. There is no reason why we should maintain a prometheus + grafana ourselves. So the link is no longer valid. I update it once metrics are ingested into the infra prometheus.

Regarding the queue size, ill send a screenshot. lowprio darwin queues can be ignored (as this includes the full queue prior to migration to the new rabbitmq, earlier this year).

10:23:48
@simon.hauser:helsinki-systems.deSimon HauserRedacted or Malformed Event10:23:57
@7c6f434c:nitro.chat7c6f434cI see, thanks. Maybe replace the link in topic with «TBD»?10:25:33
@7c6f434c:nitro.chat7c6f434c* I see, thanks. Maybe replace the link in topic with «TBD» (to avoid confusion in the meantime)?10:25:46
@simon.hauser:helsinki-systems.deSimon Hauser

nvm i am not allowed to send an image

+-----------------------------------+---------+---------+---------+
| Name                              | Ready   | Unacked | Total   |
+-----------------------------------+---------+---------+---------+
| build-inputs                      | 0       | 0       | 0       |
| build-inputs-aarch64-darwin       | 208     | 2       | 210     |
| build-inputs-aarch64-linux        | 840     | 4       | 844     |
| build-inputs-x86_64-darwin        | 16,174  | 3       | 16,177  |
| build-inputs-x86_64-linux         | 34      | 4       | 38      |
| build-results                     | 0       | 0       | 0       |
| github-dead-letters               | 0       | 0       | 0       |
| github-events-unknown             | 0       | 0       | 0       |
| logs                              | 0       | 0       | 0       |
| mass-rebuild-check-inputs         | 0       | 0       | 0       |
| mass-rebuild-check-jobs           | 0       | 1       | 1       |
+-----------------------------------+---------+---------+---------+
10:28:27
@simon.hauser:helsinki-systems.deSimon Hauseronly cole-h can change the topic10:28:53
@janne.hess:helsinki-systems.de@janne.hess:helsinki-systems.de hexa: can you do something via the moderation mot? 10:29:12
@janne.hess:helsinki-systems.de@janne.hess:helsinki-systems.de * hexa: can you do something via the moderation bot? 10:29:14
@7c6f434c:nitro.chat7c6f434cThanks!10:29:58
@0x4a6f:matrix.org[0x4A6F]Not anymore... What should be done?10:41:29
@mjolnir:nixos.orgmjolnirchanged room power levels.10:41:38
@0x4a6f:matrix.org[0x4A6F] changed the room topic to "Number of builds and evals in queue: <TBD>" from "Number of builds and evals in queue: https://ofborg.org/prometheus/graph?g0.expr=ofborg_queue_evaluator_waiting&g0.tab=1&g0.stacked=0&g0.show_exemplars=0&g0.range_input=2h&g1.expr=ofborg_queue_builder_waiting%7Barch!~%22.*-lowprior%22%7D&g1.tab=1&g1.stacked=0&g1.show_exemplars=0&g1.range_input=2h".10:43:33
@0x4a6f:matrix.org[0x4A6F]* Not anymore... What should be done? Lingering in this state, till I get an updated link.10:44:37
@7c6f434c:nitro.chat7c6f434cThanks! And OfBorg builds did work this time, thanks!13:45:28
19 Jun 2025
@fliegendewurst:matrix.orgFliegendeWurst changed their display name from FliegendeWurst to FliegendeWurst (@GPN23).12:29:23
22 Jun 2025
@ss:someonex.netSomeoneSerge (back on matrix) changed their display name from SomeoneSerge (UTC+U[-12,12]) to SomeoneSerge (Ever OOMed by Element).12:12:56
26 Jun 2025
@hexa:lossy.networkhexaI'm cleaning up the ofborg macs a bit19:51:11
@hexa:lossy.networkhexatheir git checkout grew quite a bit too large19:51:18
@hexa:lossy.networkhexaok, so I'm not sure what the holy user password is20:52:08
@hexa:lossy.networkhexait is not the one in shown in the macstadium webinterface20:52:20
@hexa:lossy.networkhexaso updating it is basically not possible20:52:26
@hexa:lossy.networkhexathe password is protected by a secure token20:53:38
@hexa:lossy.networkhexa* as the password is protected by a secure token20:53:42
@emilazy:matrix.orgemilycan you boot into recovery mode? I believe in the absence of FileVault that may be sufficient to perform user management tasks20:55:11
@hexa:lossy.networkhexado I have ssh access in recovery mode?23:15:32
27 Jun 2025
@emilazy:matrix.orgemilycan you not access the console?00:01:08

Show newer messages


Back to Room ListRoom Version: 6