!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

421 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.132 Servers

Load older messages


SenderMessageTime
3 Sep 2021
@vcunat:matrix.orgVladimír ČunátBlack Friday today?17:07:54
@baloo_:matrix.orgbaloono it's in November17:08:47
@grahamc:nixos.org@grahamc:nixos.orghttps://monitoring.nixos.org/grafana/d/wiaOmQ4nk/equinix-metal-churn?orgId=1&refresh=10s&var-instance_class=m1.xlarge.x86&var-facility=All17:20:39
@grahamc:nixos.org@grahamc:nixos.orgyou can see the churn very clearly here17:20:56
@grahamc:nixos.org@grahamc:nixos.organd one of the causes of the spot price in the top graph17:21:05
@grahamc:nixos.org@grahamc:nixos.orgams1 isn't super happy either17:23:27
@grahamc:nixos.org@grahamc:nixos.orgI'll take both out17:23:30
@grahamc:nixos.org@grahamc:nixos.orgwell now I've really stepped in it17:29:44
@grahamc:nixos.org@grahamc:nixos.orgrecreating the spot market request deleted all the existing requests (I expected this) then a bug in either the terraform provider or their API made the "create" step fail17:42:37
@grahamc:nixos.org@grahamc:nixos.orgworking with their support team18:05:02
@grahamc:nixos.org@grahamc:nixos.orglooks like we're getting hardware upgrades18:47:25
@grahamc:nixos.org@grahamc:nixos.orgI did not expect to spend half the work day on this 🙃19:05:53
4 Sep 2021
@grahamc:nixos.org@grahamc:nixos.orgokay, I finished the work to get these new classes of hardware in hydra03:01:31
@grahamc:nixos.org@grahamc:nixos.orgwent from machines with: 2x 2 x Intel Xeon E5-2650 v4 (ie: 24 cores, 2.2ghz) and 256G ram to: 1x 1 x AMD EPYC 7402P (ie: 24 cores, 2.8ghz) and 64G RAM and 1x 1 x AMD EPYC 7402P (ie: 24 cores, 2.8ghz) and 256G RAM 03:03:12
@grahamc:nixos.org@grahamc:nixos.orgthat second machine has 2x 25Gbps network connections :drool: 03:03:41
@grahamc:nixos.org@grahamc:nixos.org * went from machines with: 2x 2 x Intel Xeon E5-2650 v4 (ie: 24 cores, 2.2ghz) and 256G ram to: 1x 1 x AMD EPYC 7402P (ie: 24 cores, 2.8ghz) and 64G RAM and 1x 1 x AMD EPYC 7502P (ie: 32 cores @ 2.5Ghz) and 256G RAM 03:05:43
@lukegb:zxcvbnm.ninjalukegb (he/him)Ooh03:20:14
@lukegb:zxcvbnm.ninjalukegb (he/him) Thanks so much for your work grahamc (he/him) ❤️ 03:20:34
@grahamc:nixos.org@grahamc:nixos.org:) 03:22:29
@grahamc:nixos.org@grahamc:nixos.orgit is possible I broke the aarch64 machines while updating the channel03:43:04
@grahamc:nixos.org@grahamc:nixos.orgI'm not too stressed about that, but I'm also not likely to fix that tonight03:43:18
@domenkozar:matrix.orgDomen Kožarnixpkgs-unstable is stuck for 6 days regardless of the builds passing09:03:22
@vcunat:matrix.orgVladimír ČunátIt had lots of unfinished jobs.09:22:09
@vcunat:matrix.orgVladimír ČunátEven five days old evals still have unfinished jobs now.09:23:45
@vcunat:matrix.orgVladimír Čunátx86_64-darwin is still overloaded, due to the openssl update about a week ago (causing stdenv rebuild in there unfortunately).09:24:23
@sternenseemann:systemli.orgsternimaybe time to make a separate nixpkgs-unstable-darwin channel as well?09:36:07
@andi:kack.itandi-
In reply to @sternenseemann:systemli.org
maybe time to make a separate nixpkgs-unstable-darwin channel as well?
That would acutally also reflect the platform maturity as defined in our platform tiers RFC. (Problems on these platforms can block updates for as long as necessary to resolve the issue. is a Tier 1 privilege but it is pretty clear we are not in a situation to support it as Tier 1..)
13:12:39
@grahamc:nixos.org@grahamc:nixos.orgyar we've got some macs to add to the farm, I've gotten some of them ready for that but haven't finished yet14:04:28
@domenkozar:matrix.orgDomen Kožaryeah, macstadium donated us 5 macos machines :)16:10:03
@andi:kack.itandi-Do we have some sort of device management for those yet? I have to operate one of those as my work device and I'm hating the mutability of the whole thing already.16:47:26

Show newer messages


Back to Room ListRoom Version: 6