!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

388 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.120 Servers

Load older messages


SenderMessageTime
4 May 2025
@k900:0upti.meK900There's no formal planning16:49:54
@k900:0upti.meK900The big issues would be cache GC and scaling Hydra16:50:00
@joerg:thalheim.ioMic92It's unclear to me where this was actually deployed16:51:07
@jfly:matrix.orgJeremy Fleischman (jfly)According to https://github.com/NixOS/infra/pull/388, it sounds like it was running in Eelco's closet 16:52:08
@hexa:lossy.networkhexaRedacted or Malformed Event16:52:11
@hexa:lossy.networkhexahttps://github.com/NixOS/infra/tree/main/metrics/fastly#reports16:52:33
@jh-devv:matrix.org@jh-devv:matrix.org
In reply to @k900:0upti.me
There's no formal planning
I guess there are weekly meetups but well, more like current issues

Mainly seems like the fastly metrics are the only big one rn
17:13:13
@sinan:sinanmohd.comsinan changed their profile picture.18:36:26
@sinan:sinanmohd.comsinan changed their profile picture.18:38:37
@sinan:sinanmohd.comsinan 18:38:45
@sinan:sinanmohd.comsinan 18:38:48
@sinan:sinanmohd.comsinan 18:39:38
@sinan:sinanmohd.comsinan 18:41:08
@charles:hax0rbana.orgch joined the room.18:53:34
@nikke89:matrix.orgNikke joined the room.21:27:55
5 May 2025
@shon2929:matrix.org@shon2929:matrix.org joined the room.01:44:54
@shon2929:matrix.org@shon2929:matrix.org left the room.01:45:33
@leona:leona.isleonait feels that something is broken with hydra. eval for trunk-combined takes >1h20 (still running), build queue is super small but the scheduler and ingestion is empty according to grafana 09:32:32
@k900:0upti.meK900That's normal 09:33:19
@k900:0upti.meK900It's doing a single transaction insert of all the builds into the database first 09:33:34
@k900:0upti.meK900And only then the queue runner starts ingesting09:33:44
@k900:0upti.meK900Hydra is very sane software 09:33:59
@uep:matrix.orguepi got a few random 500's earlier today, looked like db timeouts09:36:19
@uep:matrix.orguepwasn't too surprised09:36:47
@k900:0upti.meK900My favorite part is when the insert deadlocks and gets killed 09:37:24
@k900:0upti.meK900And you know what happens next09:37:31
@k900:0upti.meK900If you guessed "tough luck run the entire eval all over again", you're correct 09:37:46
@vcunat:matrix.orgVladimír ČunátAnd that doesn't happen immediately but at the next scheduled timer (or if someone does it manually).09:43:26
@federicodschonborn:matrix.org@federicodschonborn:matrix.org changed their display name from This legally distinct plastic brick is licensed under the terms of the he/him or they/them pronouns, at your choice to This LEGO© Worm™ is licensed under the terms of the he/him or they/them pronouns, at your choice.20:48:05
6 May 2025
@federicodschonborn:matrix.org@federicodschonborn:matrix.org changed their display name from This LEGO© Worm™ is licensed under the terms of the he/him or they/them pronouns, at your choice to This LEGO® Worm™ is licensed under the terms of the he/him or they/them pronouns, at your choice.09:12:16

Show newer messages


Back to Room ListRoom Version: 6