!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

418 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.128 Servers

Load older messages


SenderMessageTime
17 May 2026
@sinan:sinanmohd.comsinan* i checked cache.nixos.org becase nixos-anywhere was lagging during substitution just noticed it was substituting from my local machine not cache.nixos.org, and my isp has shitty peering with hetzner so i routed it through my homeserver and everything went smoothly all good 👍️, still have to look at the curl issue, that was flakey18:36:33
@sinan:sinanmohd.comsinan* i checked cache.nixos.org becase it was lagging during substitution just noticed it was substituting from my local machine not cache.nixos.org, and my isp has shitty peering with hetzner so i routed it through my homeserver and everything went smoothly all good 👍️, still have to look at the curl issue, that was flakey18:37:13
@jwh4j4ez25q:matrix.orgmaurice joined the room.20:37:51
@weriomat:matrix.orgweriomat joined the room.21:20:33
@tom:dragar.deTomThe hydra.nixos.org (Github) OAuth app seems to be owned by edolstra. Possibly something that should be transferred to the GitHub Org? (from my experience that is possible but then i also haven't dealt with GitHub Enterprise)21:40:48
@hexa:lossy.networkhexa (signing key rotation when)https://github.com/NixOS/org/issues/25522:10:24
@martinhamel:matrix.org@martinhamel:matrix.org left the room.22:17:14
@tom:dragar.deTom(you probably could also use an GitHub App instead. For those you can select a Team as App managers. But since changes are seemingly quite rare and i don't know what exactly unintended consequences could be that would need more consideration and that's probably not worth it)22:23:36
@hexa:lossy.networkhexa (signing key rotation when)applied22:25:56
@tom:dragar.deTomWow, that was quick!22:38:02
@hexa:lossy.networkhexa (signing key rotation when)thanks to winterqt22:52:46
19 May 2026
@vcunat:matrix.orgvcunat

Some of the macs don't recognize my ssh keys, apparently, though most of them do. (I was running ./macs/mac-update as suggested.)

  • hetzner@sweeping-filly.mac.nixos.org
  • hetzner@maximum-snail.mac.nixos.org
  • customer@eager-heisenberg.mac.nixos.org
10:33:56
@hexa:lossy.networkhexa (signing key rotation when)they were never rolled out from nixos/infra before in that case10:37:47
@hexa:lossy.networkhexa (signing key rotation when)this will eventually sync up10:38:01
@vcunat:matrix.orgvcunatBut these aren't new, are they? They've been in the builder set for a long time, IIRC.10:39:03
@hexa:lossy.networkhexa (signing key rotation when)oh, this is in the hetzner/customer user 🤔10:39:06
@hexa:lossy.networkhexa (signing key rotation when)we don't manage keys in that user10:39:33
@hexa:lossy.networkhexa (signing key rotation when)we only manage them for root10:39:51
@vcunat:matrix.orgvcunatI mean, I'm not really interested in putting significant work into darwin stuff. I'll gladly leave that to you or others.10:40:19
@hexa:lossy.networkhexa (signing key rotation when)uh-huh10:40:53
@vcunat:matrix.orgvcunatThough it's one lock-file shared for all kinds of machines, so I guess it's better not to let the real state drift too much.10:41:14
@vcunat:matrix.orgvcunat

BTW

root@mac05.ofborg.org | error: config contains the wrong home directory for hydra-queue-builder, aborting activation
10:43:20
@hexa:lossy.networkhexa (signing key rotation when)can you file an issue for that?10:52:31
@vinceff:matrix.orgVinceff joined the room.16:47:30
@amadaluzia:4d2.orgamadaluzia changed their profile picture.20:56:29
20 May 2026
@c0ba1t:matrix.orgCobalt

Hey, I was looking into optimizations for build scheduling to account for substitution cost and was wondering how large (in terms of number/total size of derivations) the nix store was on builder nodes. This is mainly intended to make some guestimates about how expensive the query of "Which subset of a package's build graph is present on a builder node and how large (in total file size) is that subset?" is. My personal builders have varied from 250K-1M but they are almost certainly smaller/less used than the public builders.

I have looked through the data in the public Grafana instance but could not find an applicable metric. Could you please, if it is available somewhere, include this metrics in the Prometheus Instance for the NixOS Infra?

Alternatively, could someone with access to the nodes please look it up on a few builder nodes for me? It should be queriable via the sqlite DB (nix shell nixpkgs#sqlite -c sqlite3 /nix/var/nix/db/db.sqlite 'select count(*), sum(narSize) from ValidPaths' or (more expensive, without file size) by counting the store paths directly with ls -l /nix/store | wc -l)

11:18:19
@k900:0upti.meK900 Something that was brought up before is that you can just do a bloom filter over store hashes 11:32:45
@k900:0upti.meK900 It ends up on the order of a few kilobytes for a very good hit rate 11:33:06
@k900:0upti.meK900 The real problem is that requires an additional agent on the workers 11:33:22
@c0ba1t:matrix.orgCobalt

I know, I plan to do some research on this for my baechelor thesis. iirc you or hexa (signing key rotation when) broght it up when I asked for relevant issues before in the offtopic channel. The main thing for me here was to not just count store paths though but instead the total size of the store paths (as , e.g., firefox-bin is heavier than harfbuzz).

My main plan here was to make a prototype with an extra agent/nix-scheduler-hook and use the results from testing there to propose changes to hydra queue runner later.

11:38:49

Show newer messages


Back to Room ListRoom Version: 6