| 4 Apr 2023 |
 hexa | trace: lib.zip is deprecated, use lib.zipAttrsWith instead
trace: warning: lib.readPathsFromFile is deprecated, use a list instead
trace: warning: replaceChars is a deprecated alias of replaceStrings, replace usages of it with replaceStrings.
trace: `lib.nixpkgsVersion` is deprecated, use `lib.version` instead!
trace: lib.crossLists is deprecated, use lib.cartesianProductOfSets instead
trace: warning: literalExample is deprecated, use literalExpression instead, or use literalDocBook for a non-Nix description.
error: reading from file: Connection reset by peer
… while evaluating the attribute 'installPhase' of the derivation 'fgl-5.7.0.3'
| 01:54:30 |
| hexa | * trace: lib.zip is deprecated, use lib.zipAttrsWith instead
trace: warning: lib.readPathsFromFile is deprecated, use a list instead
trace: warning: replaceChars is a deprecated alias of replaceStrings, replace usages of it with replaceStrings.
trace: `lib.nixpkgsVersion` is deprecated, use `lib.version` instead!
trace: lib.crossLists is deprecated, use lib.cartesianProductOfSets instead
trace: warning: literalExample is deprecated, use literalExpression instead, or use literalDocBook for a non-Nix description.
error: reading from file: Connection reset by peer
… while evaluating the attribute 'installPhase' of the derivation 'fgl-5.7.0.3'
[...}
| 01:54:37 |
| hexa | * trace: lib.zip is deprecated, use lib.zipAttrsWith instead
trace: warning: lib.readPathsFromFile is deprecated, use a list instead
trace: warning: replaceChars is a deprecated alias of replaceStrings, replace usages of it with replaceStrings.
trace: `lib.nixpkgsVersion` is deprecated, use `lib.version` instead!
trace: lib.crossLists is deprecated, use lib.cartesianProductOfSets instead
trace: warning: literalExample is deprecated, use literalExpression instead, or use literalDocBook for a non-Nix description.
error: reading from file: Connection reset by peer
… while evaluating the attribute 'installPhase' of the derivation 'fgl-5.7.0.3'
[...]
https://gist.github.com/GrahamcOfBorg/36c45d1a59f691c9d8c5c8ad9ba50183
| 01:54:48 |
| hexa | and it doesn't respond to new eval requests | 01:54:54 |
| hexa | probably webhooks at it again | 01:54:58 |
| hexa | hote the sus line | 01:55:10 |
| hexa |
error: reading from file: Connection reset by peer
| 01:55:11 |
 cole-h | In reply to @hexa:lossy.network
and it doesn't respond to new eval requests More likely it's just in queue. There are only 7 machines, and they can run one eval at a time. | 01:55:57 |
| hexa | ah, ok | 01:56:09 |
| cole-h | (there are 4 evals waiting right now) | 01:56:24 |
| hexa | can you add a link to the prometheus dashboard into the topic? | 01:56:51 |
| hexa | ah no, you can't | 01:57:05 |
| hexa | welp | 01:57:08 |
| hexa | grahamc (he/him): some permissions fore cole-h please 🙂 | 01:57:24 |
| cole-h | https://nix.ci/prometheus/graph?g0.expr=ofborg_queue_evaluator_waiting&g0.tab=1&g0.stacked=0&g0.show_exemplars=0&g0.range_input=2h&g1.expr=ofborg_queue_builder_waiting%7Barch!~%22.*-lowprior%22%7D&g1.tab=1&g1.stacked=0&g1.show_exemplars=0&g1.range_input=1w | 01:59:01 |
| cole-h | This shows the number of waiting builds and evals. | 01:59:18 |
| cole-h | In reply to @hexa:lossy.network
trace: lib.zip is deprecated, use lib.zipAttrsWith instead
trace: warning: lib.readPathsFromFile is deprecated, use a list instead
trace: warning: replaceChars is a deprecated alias of replaceStrings, replace usages of it with replaceStrings.
trace: `lib.nixpkgsVersion` is deprecated, use `lib.version` instead!
trace: lib.crossLists is deprecated, use lib.cartesianProductOfSets instead
trace: warning: literalExample is deprecated, use literalExpression instead, or use literalDocBook for a non-Nix description.
error: reading from file: Connection reset by peer
… while evaluating the attribute 'installPhase' of the derivation 'fgl-5.7.0.3'
As for this, I'll investigate in the morning :/// | 01:59:41 |
| hexa | thanks | 02:01:13 |
|  @grahamc:nixos.orgchanged room power levels. | 02:04:05 |
| cole-h set the room topic to "Number of builds and evals in queue: https://nix.ci/prometheus/graph?g0.expr=ofborg_queue_evaluator_waiting&g0.tab=1&g0.stacked=0&g0.show_exemplars=0&g0.range_input=2h&g1.expr=ofborg_queue_builder_waiting%7Barch!~%22.*-lowprior%22%7D&g1.tab=1&g1.stacked=0&g1.show_exemplars=0&g1.range_input=1w". | 02:04:44 |
| hexa | cool! | 02:10:14 |
| hexa | the aarch64 community builder is down, so no aarch64-linux build results right now | 12:39:38 |
| cole-h | I've rebooted it | 12:52:14 |
| hexa | ssh is very slow 🤔 | 13:03:20 |
| hexa | stuck at kexinit | 13:05:12 |
| hexa | if I didn't know any better I would assume MTU 😛 | 13:05:30 |
| cole-h | [ 550.001721] mlx5_core 0001:01:00.1: wait_func:1137:(pid 18057): MODIFY_CQ(0x403) canceled on out of queue timeout.
[ 550.001723] mlx5_core 0001:01:00.0: wait_func:1137:(pid 18053): MODIFY_CQ(0x403) canceled on out of queue timeout.
[ 551.221694] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 551.227603] rcu: 60-...0: (4 GPs behind) idle=d9e4/1/0x4000000000000000 softirq=1016/1016 fqs=54368
[ 551.236726] (detected by 34, t=115624 jiffies, g=13597, q=153057 ncpus=80)
[ 551.243675] Task dump for CPU 60:
[ 551.246977] task:kworker/u160:5 state:R running task stack:0 pid:815 ppid:2 flags:0x0000000a
[ 551.256879] Workqueue: efi_rts_wq efi_call_rts
[ 551.261313] Call trace:
[ 551.263747] __switch_to+0xf0/0x170
[ 551.267226] 0xffff081f5b486ac0
[ 556.145647] mlx5_core 0001:01:00.0: wait_func:1137:(pid 18065): ACCESS_REG(0x805) canceled on out of queue timeout.
[ 558.193622] mlx5_core 0001:01:00.0: wait_func:1137:(pid 18068): ACCESS_REG(0x805) canceled on out of queue timeout.
| 13:05:34 |
| cole-h | lol | 13:05:36 |
| hexa | low entropy? | 13:05:38 |
| hexa | that call trace is magnificent | 13:06:10 |
