| 17 May 2026 |
sinan | * i checked cache.nixos.org becase nixos-anywhere was lagging during substitution
just noticed it was substituting from my local machine not cache.nixos.org, and my isp has shitty peering with hetzner
so i routed it through my homeserver and everything went smoothly
all good 👍️, still have to look at the curl issue, that was flakey | 18:36:33 |
sinan | * i checked cache.nixos.org becase it was lagging during substitution
just noticed it was substituting from my local machine not cache.nixos.org, and my isp has shitty peering with hetzner
so i routed it through my homeserver and everything went smoothly
all good 👍️, still have to look at the curl issue, that was flakey | 18:37:13 |
| maurice joined the room. | 20:37:51 |
| weriomat joined the room. | 21:20:33 |
Tom | The hydra.nixos.org (Github) OAuth app seems to be owned by edolstra. Possibly something that should be transferred to the GitHub Org? (from my experience that is possible but then i also haven't dealt with GitHub Enterprise) | 21:40:48 |
hexa (signing key rotation when) | https://github.com/NixOS/org/issues/255 | 22:10:24 |
| @martinhamel:matrix.org left the room. | 22:17:14 |
Tom | (you probably could also use an GitHub App instead. For those you can select a Team as App managers.
But since changes are seemingly quite rare and i don't know what exactly unintended consequences could be that would need more consideration and that's probably not worth it) | 22:23:36 |
hexa (signing key rotation when) | applied | 22:25:56 |
Tom | Wow, that was quick! | 22:38:02 |
hexa (signing key rotation when) | thanks to winterqt | 22:52:46 |
| 19 May 2026 |
vcunat | Some of the macs don't recognize my ssh keys, apparently, though most of them do. (I was running ./macs/mac-update as suggested.)
- hetzner@sweeping-filly.mac.nixos.org
- hetzner@maximum-snail.mac.nixos.org
- customer@eager-heisenberg.mac.nixos.org
| 10:33:56 |
hexa (signing key rotation when) | they were never rolled out from nixos/infra before in that case | 10:37:47 |
hexa (signing key rotation when) | this will eventually sync up | 10:38:01 |
vcunat | But these aren't new, are they? They've been in the builder set for a long time, IIRC. | 10:39:03 |
hexa (signing key rotation when) | oh, this is in the hetzner/customer user 🤔 | 10:39:06 |
hexa (signing key rotation when) | we don't manage keys in that user | 10:39:33 |
hexa (signing key rotation when) | we only manage them for root | 10:39:51 |
vcunat | I mean, I'm not really interested in putting significant work into darwin stuff. I'll gladly leave that to you or others. | 10:40:19 |
hexa (signing key rotation when) | uh-huh | 10:40:53 |
vcunat | Though it's one lock-file shared for all kinds of machines, so I guess it's better not to let the real state drift too much. | 10:41:14 |
vcunat | BTW
root@mac05.ofborg.org | error: config contains the wrong home directory for hydra-queue-builder, aborting activation
| 10:43:20 |
hexa (signing key rotation when) | can you file an issue for that? | 10:52:31 |
| Vinceff joined the room. | 16:47:30 |
| amadaluzia changed their profile picture. | 20:56:29 |
| 20 May 2026 |
Cobalt | Hey, I was looking into optimizations for build scheduling to account for substitution cost and was wondering how large (in terms of number/total size of derivations) the nix store was on builder nodes. This is mainly intended to make some guestimates about how expensive the query of "Which subset of a package's build graph is present on a builder node and how large (in total file size) is that subset?" is. My personal builders have varied from 250K-1M but they are almost certainly smaller/less used than the public builders.
I have looked through the data in the public Grafana instance but could not find an applicable metric. Could you please, if it is available somewhere, include this metrics in the Prometheus Instance for the NixOS Infra?
Alternatively, could someone with access to the nodes please look it up on a few builder nodes for me? It should be queriable via the sqlite DB (nix shell nixpkgs#sqlite -c sqlite3 /nix/var/nix/db/db.sqlite 'select count(*), sum(narSize) from ValidPaths' or (more expensive, without file size) by counting the store paths directly with ls -l /nix/store | wc -l)
| 11:18:19 |
K900 | Something that was brought up before is that you can just do a bloom filter over store hashes | 11:32:45 |
K900 | It ends up on the order of a few kilobytes for a very good hit rate | 11:33:06 |
K900 | The real problem is that requires an additional agent on the workers | 11:33:22 |
Cobalt | I know, I plan to do some research on this for my baechelor thesis. iirc you or hexa (signing key rotation when) broght it up when I asked for relevant issues before in the offtopic channel. The main thing for me here was to not just count store paths though but instead the total size of the store paths (as , e.g., firefox-bin is heavier than harfbuzz).
My main plan here was to make a prototype with an extra agent/nix-scheduler-hook and use the results from testing there to propose changes to hydra queue runner later.
| 11:38:49 |