!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

415 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.130 Servers

Load older messages


SenderMessageTime
9 May 2026
@emilazy:matrix.orgemily and obviously I don't know how the new queue runner will handle uploads (maybe John Ericson does) 20:21:44
@hexa:lossy.networkhexa Simon Hauser would know 20:22:08
@emilazy:matrix.orgemilyit seems like just increasing the number of retries in the Nix config specifically used by the queue runner would likely mitigate this problem in practice for now20:22:13
@hexa:lossy.networkhexafair enough20:22:29
@hexa:lossy.networkhexalet's say 32 instead of 1024 though20:22:37
@Ericson2314:matrix.orgJohn EricsonI do agree with that sort of thing20:23:49
@Ericson2314:matrix.orgJohn Ericson(well in the CA case, it would be the build trace entry, but I digress :))20:24:04
@emilazy:matrix.orgemilyin the CA case the Darwin builds will be broken 100% of the time so there'll be much less debugging required to find the root cause :P20:24:37
@Ericson2314:matrix.orgJohn Ericsonno more automatic rewriting in CA soon! :)20:26:12
@emilazy:matrix.orgemilyhow is that going to work?20:26:42
@Ericson2314:matrix.orgJohn Ericsonhttps://github.com/NixOS/nix/pull/15793 see what Artemis has been working on20:27:51
@Ericson2314:matrix.orgJohn Ericson(note that the use of the exing protocol vs a new simpler protocol is provisional)20:28:07
@hexa:lossy.networkhexahow annoying would it be to move the S3 bucket to Europe btw 😆20:28:54
@emilazy:matrix.orgemilyI don't see how this solves every dylib on Darwin encoding its own path?20:29:51
@emilazy:matrix.orgemily "The builder can learn the calculated content-addressed path of one of its outputs before creating others" sounds like it can only prevent breaking cross-output references (and since you usually need to know all paths upfront at ./configure time probably not even then) 20:30:12
@emilazy:matrix.orgemilybut offtopic, anyway :)20:30:14
@squalus0:matrix.org@squalus0:matrix.org left the room.21:21:39
10 May 2026
@vcunat:matrix.orgVladimír Čunát
In reply to @emilazy:matrix.org
right, okay. I don't know exactly why it happens but I know vcunat has mentioned it happening multiple times before.
To me the behavior looked like insufficient deduplication of the situations where different hydra jobs built the same derivation, e.g. on different jobsets. It seemed easy to happen. We're helped by the ratio of job count to build times usually, as the duplication only happens concurrently. And for linux we get helped by the builder pool being small and split along the big-parallel axis, so often the duplicate got scheduled to the same machine and deduplicated locally by nix in there.
06:27:16
@me:m4rc3l.deMarcel joined the room.12:45:09
@Ericson2314:matrix.orgJohn EricsonI think we should disable fallback paths15:13:06
@Ericson2314:matrix.orgJohn EricsonIt seems like this feature is not being helpful at all15:13:20
@Ericson2314:matrix.orgJohn EricsonAnd it would be better to fail or delete the outputs first 15:13:38
@k900:0upti.meK900 Problem with delete the outputs is they may be in use 15:14:09
@k900:0upti.meK900 Problem with fail is Hydra needs to be smart enough to handle this 15:14:20
@k900:0upti.meK900 But ideally yes we should fail 15:14:25
@hexa:lossy.networkhexanot sure who restarted ip2unix on nixos/unstable, but it's stuck on a test17:13:30
@Ericson2314:matrix.orgJohn Ericsonthere will be temp roots if they are in use17:13:29
@hexa:lossy.networkhexahttps://hydra.nixos.org/build/326816958/nixlog/5/tail17:13:31
@hexa:lossy.networkhexa
root      374239  0.0  0.0 638608 18368 ?        Ssl  May06   2:47 nix-daemon --daemon
root     3020110  0.0  0.0 2137748 27724 ?       Ssl  14:32   0:01  \_ nix-daemon 3016813
nixbld13 3022134  0.0  0.0   4568  3376 ?        Ss   14:32   0:00  |   \_ bash -e /nix/store/l622p70vy8k5sh7y5wizi5f2mic6ynpg-source-stdenv.sh /nix/store/shkw4qm9qcw5sc5n1k5jznc83ny02r39-default-builder.sh
nixbld13 3038056  0.0  0.0  50640 40648 ?        S    14:32   0:02  |       \_ /nix/store/0r6k8xa2kgqyp3r4v2w7yrb80ma2iawm-python3-3.13.12/bin/python3.13 /nix/store/vn448hbsgbh0ahk1bjv2l16f5lhambiq-meson-1.10.2/bin/meson test --no-rebuild --print-errorlogs --max-lines=1000000 --timeout-multiplier=0
nixbld13 3038254  0.0  0.0  37052 32488 ?        Ss   14:32   0:00  |           \_ /nix/store/0r6k8xa2kgqyp3r4v2w7yrb80ma2iawm-python3-3.13.12/bin/python3.13 /nix/store/wavm4b318kvk7xqy7qfam86i0r8znz1n-python3.13-pytest-9.0.2/bin/.pytest-wrapped -p no:cacheprovider --ip2unix-path=/build/source/build/ip2unix --libip2unix-path=/build/source/build/libip2unix.so --helper-accept-no-peer-addr=/build/source/build/tests/helpers/helper_accept_no_peer_addr --timeout=60 --systemd-support --systemd-sa-path=/nix/store/9rpism89x6lyjcwzzkp6kana25rs03nn-systemd-260.1/bin/systemd-socket-activate
nixbld13 3038432  0.0  0.0   7436  3088 ?        S    14:32   0:00  |               \_ /build/source/build/tests/helpers/helper_accept_no_peer_addr
17:13:53
@Ericson2314:matrix.orgJohn Ericsonreminds me I really want to run all test suites in separate derivations17:13:54

Show newer messages


Back to Room ListRoom Version: 6