!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

479 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.149 Servers

Load older messages


SenderMessageTime
18 Jun 2026
@no-mood:matrix.orgno-mood joined the room.12:12:45
@k900:0upti.meK900 Do we know what's causing the corruption? 12:13:06
@k900:0upti.meK900 If we don't, maybe we should just stop the presses 12:13:23
@joerg:thalheim.ioMic92So the queue runner was re-using deamon connections that were in a undefined state. I saw that some queue builder logs where showing that deamon queries were return garbage that must have been still there from other connections. This has likely caused that path-info queries got the wrong result12:17:53
@magic_rb:matrix.redalder.orgmagic_rb joined the room.12:17:56
@joerg:thalheim.ioMic92Now I replaced all read-only store queries by going to the db directly and open a new daemon connection for each other operation that actually requires mutation12:18:50
@tbaldwin:matrix.orgTrent Baldwin joined the room.12:20:16
@cyborgpotato:glia.clubcyborgpotato joined the room.12:32:51
@amboss_mann:matrix.orgAmboss_Mann joined the room.12:51:44
@liamthexpl0rer:l14mx.deL14m joined the room.12:59:50
@vcunat:matrix.orgVladimír Čunát

Uh, while these issues are solved now probably (we had direct communication), there's some more blocking (all?) channels:

nix-support/hydra-build-products doesn't work well anymore, somehow.

13:13:04
@vcunat:matrix.orgVladimír ČunátHere we're missing the "build products": https://hydra.nixos.org/build/331662168#tabs-summary like it was e.g. in https://hydra.nixos.org/build/331523718#tabs-summary13:13:40
@vcunat:matrix.orgVladimír ČunátWhen I fetch the derivation, the correct file does seem to be in there.13:14:01
@vcunat:matrix.orgVladimír Čunát

The channel scripts then fail with

Jun 18 13:12:11 pluto update-nixos-26.05-small-start[789977]: No product could be selected for nixos.options, with type json-br at [...]

13:14:52
@tom:toostveen.nltom joined the room.13:18:28
@joerg:thalheim.ioMic92 vcunat: new compression is causing this I think. 13:20:23
@vcunat:matrix.orgVladimír ČunátIt seems to work on the last nixos-unstable-small, though: https://hydra.nixos.org/build/33166687713:20:11
@joerg:thalheim.ioMic92is this channel scripts?13:20:37
@vcunat:matrix.orgVladimír ČunátIt shows wrong in the Hydra UI already.13:20:54
@vcunat:matrix.orgVladimír Čunát* It shows wrong in the Hydra's web UI already.13:20:57
@vcunat:matrix.orgVladimír ČunátThe scripts just react to missing it in there.13:21:20
@joerg:thalheim.ioMic92Okay. Working on it13:31:49
@r-burns:matrix.orgr-burnsHappy to help out with any rote tasks that are too difficult for an LLM but still simple enough for a moderately intelligent human - let me know if I can assist in any way.13:33:52
@deraffe:matrix.orgderaffe joined the room.13:39:58
@vcunat:matrix.orgVladimír ČunátI bumped https://hydra.nixos.org/jobset/nixos/unstable-small to see if at least other parts of the pipeline to update -small channels works OK.13:40:43
@vcunat:matrix.orgVladimír ČunátAs this problem is OK, somehow.13:41:14
@arcayr:mischief.expertarcayr changed their profile picture.13:42:12
@joerg:thalheim.ioMic92I might have the fix. Just double checking with the system13:42:19
@hexa:lossy.networkhexa can we clarify what "Receiving outputs" means for presigned uploads? Is it then uploading? 13:47:42
@joerg:thalheim.ioMic92

The flow is like this:

  1. step_status = ReceivingOutputs
  2. upload_nars_presigned() on the queue builder
  3. step_status = PostProcessing
13:50:10

Show newer messages


Back to Room ListRoom Version: 6