!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

409 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.127 Servers

Load older messages


SenderMessageTime
9 May 2026
@Ericson2314:matrix.orgJohn EricsonI do agree with that sort of thing20:23:49
@Ericson2314:matrix.orgJohn Ericson(well in the CA case, it would be the build trace entry, but I digress :))20:24:04
@emilazy:matrix.orgemilyin the CA case the Darwin builds will be broken 100% of the time so there'll be much less debugging required to find the root cause :P20:24:37
@Ericson2314:matrix.orgJohn Ericsonno more automatic rewriting in CA soon! :)20:26:12
@emilazy:matrix.orgemilyhow is that going to work?20:26:42
@Ericson2314:matrix.orgJohn Ericsonhttps://github.com/NixOS/nix/pull/15793 see what Artemis has been working on20:27:51
@Ericson2314:matrix.orgJohn Ericson(note that the use of the exing protocol vs a new simpler protocol is provisional)20:28:07
@hexa:lossy.networkhexa (signing key rotation when)how annoying would it be to move the S3 bucket to Europe btw 😆20:28:54
@emilazy:matrix.orgemilyI don't see how this solves every dylib on Darwin encoding its own path?20:29:51
@emilazy:matrix.orgemily "The builder can learn the calculated content-addressed path of one of its outputs before creating others" sounds like it can only prevent breaking cross-output references (and since you usually need to know all paths upfront at ./configure time probably not even then) 20:30:12
@emilazy:matrix.orgemilybut offtopic, anyway :)20:30:14
@squalus0:matrix.org@squalus0:matrix.org left the room.21:21:39
10 May 2026
@vcunat:matrix.orgvcunat
In reply to @emilazy:matrix.org
right, okay. I don't know exactly why it happens but I know vcunat has mentioned it happening multiple times before.
To me the behavior looked like insufficient deduplication of the situations where different hydra jobs built the same derivation, e.g. on different jobsets. It seemed easy to happen. We're helped by the ratio of job count to build times usually, as the duplication only happens concurrently. And for linux we get helped by the builder pool being small and split along the big-parallel axis, so often the duplicate got scheduled to the same machine and deduplicated locally by nix in there.
06:27:16
@me:m4rc3l.deMarcel joined the room.12:45:09
@Ericson2314:matrix.orgJohn EricsonI think we should disable fallback paths15:13:06
@Ericson2314:matrix.orgJohn EricsonIt seems like this feature is not being helpful at all15:13:20
@Ericson2314:matrix.orgJohn EricsonAnd it would be better to fail or delete the outputs first 15:13:38
@k900:0upti.meK900 Problem with delete the outputs is they may be in use 15:14:09
@k900:0upti.meK900 Problem with fail is Hydra needs to be smart enough to handle this 15:14:20
@k900:0upti.meK900 But ideally yes we should fail 15:14:25
@hexa:lossy.networkhexa (signing key rotation when)not sure who restarted ip2unix on nixos/unstable, but it's stuck on a test17:13:30
@Ericson2314:matrix.orgJohn Ericsonthere will be temp roots if they are in use17:13:29
@hexa:lossy.networkhexa (signing key rotation when)https://hydra.nixos.org/build/326816958/nixlog/5/tail17:13:31
@hexa:lossy.networkhexa (signing key rotation when)
root      374239  0.0  0.0 638608 18368 ?        Ssl  May06   2:47 nix-daemon --daemon
root     3020110  0.0  0.0 2137748 27724 ?       Ssl  14:32   0:01  \_ nix-daemon 3016813
nixbld13 3022134  0.0  0.0   4568  3376 ?        Ss   14:32   0:00  |   \_ bash -e /nix/store/l622p70vy8k5sh7y5wizi5f2mic6ynpg-source-stdenv.sh /nix/store/shkw4qm9qcw5sc5n1k5jznc83ny02r39-default-builder.sh
nixbld13 3038056  0.0  0.0  50640 40648 ?        S    14:32   0:02  |       \_ /nix/store/0r6k8xa2kgqyp3r4v2w7yrb80ma2iawm-python3-3.13.12/bin/python3.13 /nix/store/vn448hbsgbh0ahk1bjv2l16f5lhambiq-meson-1.10.2/bin/meson test --no-rebuild --print-errorlogs --max-lines=1000000 --timeout-multiplier=0
nixbld13 3038254  0.0  0.0  37052 32488 ?        Ss   14:32   0:00  |           \_ /nix/store/0r6k8xa2kgqyp3r4v2w7yrb80ma2iawm-python3-3.13.12/bin/python3.13 /nix/store/wavm4b318kvk7xqy7qfam86i0r8znz1n-python3.13-pytest-9.0.2/bin/.pytest-wrapped -p no:cacheprovider --ip2unix-path=/build/source/build/ip2unix --libip2unix-path=/build/source/build/libip2unix.so --helper-accept-no-peer-addr=/build/source/build/tests/helpers/helper_accept_no_peer_addr --timeout=60 --systemd-support --systemd-sa-path=/nix/store/9rpism89x6lyjcwzzkp6kana25rs03nn-systemd-260.1/bin/systemd-socket-activate
nixbld13 3038432  0.0  0.0   7436  3088 ?        S    14:32   0:00  |               \_ /build/source/build/tests/helpers/helper_accept_no_peer_addr
17:13:53
@Ericson2314:matrix.orgJohn Ericsonreminds me I really want to run all test suites in separate derivations17:13:54
@Ericson2314:matrix.orgJohn Ericsonthough also, the ability to pipeline outputs separately with the new RPC thing can ameliorate actually needing to split17:14:48
@Ericson2314:matrix.orgJohn Ericsonjust have a "slow output" for tests17:15:09
@vcunat:matrix.orgvcunatIt's stuck repeatedly in the past week. Probably always. 10 hours and more, until I cancel.18:09:09
@vcunat:matrix.orgvcunatnixos-unstable is blocked anyway by https://hydra.nixos.org/build/328803284/nixlog/1/tail18:10:25
@vcunat:matrix.orgvcunat(and restarting won't help)18:10:40

Show newer messages


Back to Room ListRoom Version: 6