!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

422 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.132 Servers

Load older messages


SenderMessageTime
12 May 2026
@hexa:lossy.networkhexa
May 12 13:55:45 pluto systemd[1]: Starting Update Channel nixos-unstable...
May 12 13:55:55 pluto update-nixos-unstable-start[500454]:  $ git fetch origin da5ad661ba4e5ef59ba743f0d112cbc30e474f32 >&2
May 12 13:55:55 pluto update-nixos-unstable-start[500477]: From https://github.com/NixOS/nixpkgs
May 12 13:55:55 pluto update-nixos-unstable-start[500477]:  * branch                      da5ad661ba4e5ef59ba743f0d112cbc30e474f32 -> FETCH_HEAD
May 12 13:55:55 pluto update-nixos-unstable-start[500454]: Release information:
May 12 13:55:55 pluto update-nixos-unstable-start[500454]:  - release is: nixos-26.05pre995699.da5ad661ba4e (build 328830334)
May 12 13:55:55 pluto update-nixos-unstable-start[500454]:  - eval is: 1825250
May 12 13:55:55 pluto update-nixos-unstable-start[500454]:  - prefix is: nixos/unstable/nixos-26.05pre995699.da5ad661ba4e
May 12 13:55:55 pluto update-nixos-unstable-start[500454]:  - Git commit is: da5ad661ba4e5ef59ba743f0d112cbc30e474f32
May 12 13:55:55 pluto update-nixos-unstable-start[500454]:  - Git commit date is: 2026-05-10 19:57:52 UTC
May 12 13:55:56 pluto update-nixos-unstable-start[500454]: previous release is https://releases.nixos.org/nixos/unstable/nixos-26.05pre992384.549bd84d6279
May 12 13:56:07 pluto update-nixos-unstable-start[500454]: downloading /nix/store/6kq7jn3mqq8w2x9bjgbyd698pir2ha0l-nixos-channel-26.05pre995699.da5ad661ba4e/tarballs/nixos-26.05pre995699.da5ad661ba4e.tar.xz to /home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/nixexprs.tar.xz...
May 12 13:56:07 pluto update-nixos-unstable-start[500454]:  $ NIX_REMOTE=s3://nix-cache nix --experimental-features nix-command store cat '/nix/store/6kq7jn3mqq8w2x9bjgbyd698pir2ha0l-nixos-channel-26.05pre995699.da5ad661ba4e/tarballs/nixos-26.05pre995699.da5ad661ba4e.tar.xz' > '/home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/nixexprs.tar.xz.tmp'
May 12 13:56:23 pluto update-nixos-unstable-start[500454]: downloading /nix/store/hbl8zmq0sp4m25qz8gr41gd0zryzz3fp-nixpkgs-tarball-26.05pre995699.da5ad661ba4e/packages.json.br to /home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/packages.json.br...
May 12 13:56:23 pluto update-nixos-unstable-start[500454]:  $ NIX_REMOTE=s3://nix-cache nix --experimental-features nix-command store cat '/nix/store/hbl8zmq0sp4m25qz8gr41gd0zryzz3fp-nixpkgs-tarball-26.05pre995699.da5ad661ba4e/packages.json.br' > '/home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/packages.json.br.tmp'
May 12 13:56:39 pluto update-nixos-unstable-start[500454]: downloading /nix/store/6qcvsf95rk0whh1paf1qz0sk6k2pkgy0-options.json/share/doc/nixos/options.json.br to /home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/options.json.br...
May 12 13:56:39 pluto update-nixos-unstable-start[500454]:  $ NIX_REMOTE=s3://nix-cache nix --experimental-features nix-command store cat '/nix/store/6qcvsf95rk0whh1paf1qz0sk6k2pkgy0-options.json/share/doc/nixos/options.json.br' > '/home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/options.json.br.tmp'
May 12 13:56:51 pluto update-nixos-unstable-start[500454]: downloading /nix/store/8ffmwwc9h1v9wkadrvjznxkw3b4895aw-nixos-minimal-26.05pre995699.da5ad661ba4e-aarch64-linux.iso/iso/nixos-minimal-26.05pre995699.da5ad661ba4e-aarch64-linux.iso to /home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/nixos-minimal-26.05pre995699.da5ad661ba4e-aarch64-linux.iso...
May 12 13:56:51 pluto update-nixos-unstable-start[500454]:  $ NIX_REMOTE=s3://nix-cache nix --experimental-features nix-command store cat '/nix/store/8ffmwwc9h1v9wkadrvjznxkw3b4895aw-nixos-minimal-26.05pre995699.da5ad661ba4e-aarch64-linux.iso/iso/nixos-minimal-26.05pre995699.da5ad661ba4e-aarch64-linux.iso' > '/home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/nixos-minimal-26.05pre995699.da5ad661ba4e-aarch64-linux.iso.tmp'
May 12 13:58:53 pluto update-nixos-unstable-start[500454]: downloading /nix/store/2zfpap3a8934dz0w32hpzj5v94ggmwaj-nixos-minimal-26.05pre995699.da5ad661ba4e-x86_64-linux.iso/iso/nixos-minimal-26.05pre995699.da5ad661ba4e-x86_64-linux.iso to /home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/nixos-minimal-26.05pre995699.da5ad661ba4e-x86_64-linux.iso...
May 12 13:58:53 pluto update-nixos-unstable-start[500454]:  $ NIX_REMOTE=s3://nix-cache nix --experimental-features nix-command store cat '/nix/store/2zfpap3a8934dz0w32hpzj5v94ggmwaj-nixos-minimal-26.05pre995699.da5ad661ba4e-x86_64-linux.iso/iso/nixos-minimal-26.05pre995699.da5ad661ba4e-x86_64-linux.iso' > '/home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/nixos-minimal-26.05pre995699.da5ad661ba4e-x86_64-linux.iso.tmp'
May 12 14:00:51 pluto update-nixos-unstable-start[500454]: downloading /nix/store/y2mj621m81xl2gbfmhw6cqad5kv3bngm-nixos-graphical-26.05pre995699.da5ad661ba4e-aarch64-linux.iso/iso/nixos-graphical-26.05pre995699.da5ad661ba4e-aarch64-linux.iso to /home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/nixos-graphical-26.05pre995699.da5ad661ba4e-aarch64-linux.iso...
May 12 14:00:51 pluto update-nixos-unstable-start[500454]:  $ NIX_REMOTE=s3://nix-cache nix --experimental-features nix-command store cat '/nix/store/y2mj621m81xl2gbfmhw6cqad5kv3bngm-nixos-graphical-26.05pre995699.da5ad661ba4e-aarch64-linux.iso/iso/nixos-graphical-26.05pre995699.da5ad661ba4e-aarch64-linux.iso' > '/home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/nixos-graphical-26.05pre995699.da5ad661ba4e-aarch64-linux.iso.tmp'
May 12 14:04:47 pluto update-nixos-unstable-start[500454]: downloading /nix/store/g62l1p9d40l9d2wrmcimyc61fw7dr3l6-nixos-graphical-26.05pre995699.da5ad661ba4e-x86_64-linux.iso/iso/nixos-graphical-26.05pre995699.da5ad661ba4e-x86_64-linux.iso to /home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/nixos-graphical-26.05pre995699.da5ad661ba4e-x86_64-linux.iso...
May 12 14:04:47 pluto update-nixos-unstable-start[500454]:  $ NIX_REMOTE=s3://nix-cache nix --experimental-features nix-command store cat '/nix/store/g62l1p9d40l9d2wrmcimyc61fw7dr3l6-nixos-graphical-26.05pre995699.da5ad661ba4e-x86_64-linux.iso/iso/nixos-graphical-26.05pre995699.da5ad661ba4e-x86_64-linux.iso' > '/home/hydra-mirror/scratch/release-nixos-unstable/nixos-26.05pre995699.da5ad661ba4e/nixos-graphical-26.05pre995699.da5ad661ba4e-x86_64-linux.iso.tmp'
14:38:15
@hexa:lossy.networkhexa
Download
14:38:34
@hexa:lossy.networkhexaprobably going to restart, that looks pretty much suck14:39:04
@hexa:lossy.networkhexa

30m

14:39:11
@hexa:lossy.networkhexaRedacted or Malformed Event14:39:14
@hexa:lossy.networkhexaRedacted or Malformed Event14:39:45
@vcunat:matrix.orgVladimír Čunát I see constant network traffic flowing on the pluto machine. 14:40:13
@hexa:lossy.networkhexa
Download
14:41:49
@hexa:lossy.networkhexayeah, 5 Mbps14:41:54
@hexa:lossy.networkhexacool.14:41:56
@vcunat:matrix.orgVladimír ČunátWe could restart and see if it gets better, yeah 😅14:42:25
@hexa:lossy.networkhexaI did14:42:28
@harinn:matrix.orgHarinn joined the room.18:14:42
14 May 2026
@hexa:lossy.networkhexathat is tomorrowish01:28:32
@hexa:lossy.networkhexa
hetzner@intense-heron.mac.nixos.org |  3:28  up 48 days,  8:07, 0 users, load averages: 15.91 14.71 11.59
customer@eager-heisenberg.mac.nixos.org |  1:28  up 28 days, 13:38, 1 user, load averages: 7.24 8.43 8.04
customer@kind-lumiere.mac.nixos.org |  1:28  up 40 days,  9:47, 1 user, load averages: 4.58 6.38 6.41
hetzner@growing-jennet.mac.nixos.org |  3:28  up 20 days, 22:27, 0 users, load averages: 5.80 8.05 9.15
hetzner@enormous-catfish.mac.nixos.org |  3:28  up 40 days,  9:54, 0 users, load averages: 5.01 7.72 7.86
hetzner@sweeping-filly.mac.nixos.org |  3:28  up 40 days,  9:59, 0 users, load averages: 3.97 5.49 6.28
hetzner@maximum-snail.mac.nixos.org |  3:28  up 40 days,  9:55, 0 users, load averages: 8.13 7.53 7.29
root@norwegian-blue.mac.nixos.org |  3:28  up 2 days, 16:39, 1 user, load averages: 2.55 4.79 5.16
01:28:53
@tjni:matrix.org@tjni:matrix.org left the room.04:57:49
@vcunat:matrix.orgVladimír Čunát

Channels are blocked.

remote: Personal access tokens (classic) are forbidden from accessing this repository.

https://github.com/NixOS/org/issues/247#issuecomment-4447783439

09:38:48
@emilazy:matrix.orgemilyI did ping infra team on that issue weeks ago after doing a brief review myself 😅13:32:02
@emilazy:matrix.orgemilywe should probably roll back for now until the uses can be fixed13:32:15
@emilazy:matrix.orgemilythe channel scripts should be using a GitHub app like CI/rfc39/etc. do, most likely13:33:40
@emilazy:matrix.orgemilyIIRC it looked to me like the channel scripts used an SSH key for the Git push btw, what is the token in question used for?13:53:32
@vcunat:matrix.orgVladimír ČunátWell, I did not know this stuff. Just tried to diagnose the issue quickly.14:22:55
@vcunat:matrix.orgVladimír Čunát

Which pointed me to

GIT_DIR=$dir git config credential.helper 'store --file=${config.age.secrets.hydra-mirror-git-credentials.path}'
14:37:47
@hexa:lossy.networkhexathis is easily fixed14:50:56
@hexa:lossy.networkhexawe'll go for an ssh key this time, I think14:56:06
@emilazy:matrix.orgemilyyeah, making it use an app is probably good for the long term to scope the permissions further but SSH key will at least restrict it down to Git ops15:14:48
@emilazy:matrix.orgemilysorry for missing that when looking through the infra repo15:15:18
@emilazy:matrix.orgemilyI'll look through the other reports in more detail later15:16:28
@emilazy:matrix.orgemilybut I guess this was the only thing noticed for official infra?15:16:38
@hexa:lossy.networkhexaI didn't check, because I assumed you did15:18:12

Show newer messages


Back to Room ListRoom Version: 6