!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

391 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.120 Servers

Load older messages


SenderMessageTime
11 Jan 2025
@hexa:lossy.networkhexaintense-heron causing eofs on builds15:39:59
@hexa:lossy.networkhexafull disk15:41:19
@vcunat:matrix.orgVladimír Čunát Why should trunk be mostly downloads? 16:01:36
@sternenseemann:systemli.orgsternihm you're right we haven't had a staging merge recently16:02:09
@sternenseemann:systemli.orgsterniI assumed something like that had happened since I saw always 10k queued builds a few days ago16:02:29
@vcunat:matrix.orgVladimír ČunátHalf an hour for kernel, around five hours for chromium. I guess that's OK latency now.16:03:02
@vcunat:matrix.orgVladimír Čunát As for release-24.11, it could be finished relatively fast now that I bumped it (after its tested job succeeded). Even if some work is relatively cheap, prioritization still matters. 16:41:22
12 Jan 2025
@hexa:lossy.networkhexahttps://github.com/NixOS/infra/issues/53400:33:22
11 Jan 2025
@oak:universumi.fioak 🏳️‍🌈♥️ changed their profile picture.16:45:53
12 Jan 2025
@hexa:lossy.networkhexaThe database on haumea.nixos.org failed an automatic vacuum, because it ran out of disk space.00:33:47
11 Jan 2025
@oak:universumi.fioak 🏳️‍🌈♥️ changed their profile picture.16:46:55
12 Jan 2025
@hexa:lossy.networkhexaI've removed some expensive snapshots and started a manual full vacuum, so we can hopefully save some space.00:34:21
@xv1:matrix.org@xv1:matrix.org joined the room.02:51:44
@xv1:matrix.org@xv1:matrix.org left the room.03:03:56
@hexa:lossy.networkhexathis should be the ETA 🕖️03:09:33
@hexa:lossy.networkhexavacuum done05:26:50
@vcunat:matrix.orgVladimír Čunát

zrepl got into a stuck state, apparently.

From zrepl status I gather that some snapshot got removed too early for it to recover. Sending a "reset" signal doesn't seem to help.

08:12:42
@hexa:lossy.networkhexathere was not enough disk space to keep them around14:13:31
@hexa:lossy.networkhexathere is a partically received snapshot on the receiving end14:15:46
@hexa:lossy.networkhexahttps://github.com/NixOS/infra/issues/534#issuecomment-258575567014:25:49
13 Jan 2025
@hexa:lossy.networkhexaI picked two patches that enables streaming of files from the store, instead of buffering them first00:29:39
@hexa:lossy.networkhexathat works fairly well for the minimal isos00:33:12
@hexa:lossy.networkhexaline rate from a hetzner cloud vm00:33:24
@hexa:lossy.networkhexabut the gnome/plasma ISOs are waiting for a response still00:33:46
@hexa:lossy.networkhexaah, there it is00:40:08
@xin:xiny.lixin joined the room.02:55:40
@hexa:lossy.networkhexaluckily we have none of these machines 🙈 https://status.hetzner.com/incident/7fae9cca-b38c-4154-8a27-14e6dfea5c1e13:50:29
@hexa:lossy.networkhexathey're replacing mainboards across all AX42, AX52 and AX102 due to a design error and estimate it will take them a full year to complete this task13:51:15
@kranzes:matrix.orgIlan Joselevich (Kranzes) I had a motherboard on an AX102 die on me last month 14:11:47
14 Jan 2025
@dgrig:erethon.comdgrig

Hello! I've been asked to setup https://github.com/gristlabs/grist-core for use by the Foundation and NGI in foundation owned infra. Is there someone I can talk to about this? I can also join the next sync call if those are happening still. Let me know if you prefer me to create an issue to properly track this discussion.

CC fricklerhandwerk

09:30:29

Show newer messages


Back to Room ListRoom Version: 6