!zghijEASpYQWYFzriI:nixos.org

Hydra

390 Members
114 Servers

Load older messages


SenderMessageTime
11 May 2026
@janne:hess.oooJanne* not sure that even matters anymore tbf19:22:56
@janne:hess.oooJanne* not sure that even matters anymore tbh19:23:02
@Ericson2314:matrix.orgJohn Ericson....because you think the code is all slop and never gonna work.....?19:24:37
@frooastside:wolkeneis.devFrooastside joined the room.19:28:10
@janne:hess.oooJanne

Yes but no. I don't doubt whether the code will work, it most likely will with a probability of failure compared to a human. The actual issue I have with the entire topic is not the LLM but the way it's handled and applied. There are multiple issues I see, all connected to maintainability.

  • There is currently a lot of work going on that seems unfocused to the actual goal of getting the new qr to prod and each of these changes introduces additional risks of bugs
  • Most of the code is self-merged with only one person (I hope) ever seeing that code and no additional person knowing any of the details without clicking through the 100 commits that have happened since the qr. Which is getting less reasonable with the current pace of new commits and PRs. Also, there is no actual rush for most of these changes. What matters right now is getting the qr to prod so we can have take a look at regressions and fix them after such a huge change
  • There have been surprising changes where I would have expected the entire team to be consulted like introducing a toml formatter that is in my perspective very subpar
  • We have very limited human resources in the project. Before me there was only graham (and someone else, sorry I forgot the name) and I don't see how permanent the current amount of people is. I think (don't pin me on this statement) I have been the longest maintainer of the software and I assume that after right now, it will be back to me again (if I decide this works for me) and the others following different projects (which is fair, there is no lifetime requirement) - leaving me with a codebase I have no idea how it works (like back when I started) in a state only a LLM can understand and which I am not willing to spend my own money on the amount of tokens it requires. Unfortunately, there has to be someone maintaing this software and both my past experience and what I think people I have worked with expect of me gets me into the situation where I will be doing the required work
19:53:04
@Ericson2314:matrix.orgJohn EricsonI would liek to work more closely but how often have you been checking matrix etc?19:56:09
@Ericson2314:matrix.orgJohn EricsonRight now I have a limitted time budget to get a lot of things done, including fixing preexisting issues19:56:26
@Ericson2314:matrix.orgJohn Ericsonwhich again is my focus right now, I tend to solve all problems with more refactoring19:56:54
@Ericson2314:matrix.orgJohn Ericson(we're pretty well done on the new feature work, actually)19:57:25
@Ericson2314:matrix.orgJohn Ericsonthere has been no unattended LLMs, I read code very quickly19:58:00
@Ericson2314:matrix.orgJohn Ericsonand my knowledge of the codebase trends up not done, as I choose to take the risk of starting with the rust code I was not familiar with19:58:34
@Ericson2314:matrix.orgJohn Ericsonwe can rip out the toml formatting if you don't like that, but that is the most trivial bikeshed thing19:58:53
@Ericson2314:matrix.orgJohn Ericsonnot load-berring for the overall health of the project19:59:06
@frooastside:wolkeneis.devFrooastside

Hello, I am not sure if this is the right place to ask, but I set up a hydra instance on my server and it all works so far but I am a bit confused by the architecture. I used the newer hydra-queue-builder and runner that came from helsinki-systems and when garbage collection ran on the builder, the next time it builds something, it tries to build everything starting from bootsrap stage 1. I enabled substitutions and the hydra-queue-runner says something like this:

hydra-queue-runner[84836]: 2026-05-11T19:17:16.161281Z  INFO queue_monitor_loop:get_queued_builds:process_new_builds:create_build:create_step:create_step: hydra_queue_runner::state: create_step: llw11xba2z4kfiyka62yhfjc75fys75x-bash-5.3p3.drv already finished (outputs in store), skipping build_id=1 build_id=1 drv_path=a97x63dkns7qsza17rbpgkmn6k9f65sf-rp4-image-hydra.drv build_id=1 build_id=1 drv_path=a97x63dkns7qsza17rbpgkmn6k9f65sf-rp4-image-hydra.drv build_id=1 drv_path=llw11xba2z4kfiyka62yhfjc75fys75x-bash-5.3p3.drv

so it already is in the s3 cache (it got build by the builder earlier) and the queue runner knows that. the queue-builder has the cache enabled as a subsituter but doesnt use it, is this intended?

20:01:22
@Ericson2314:matrix.orgJohn Ericson Frooastside: this is the right spot 20:02:49
@Ericson2314:matrix.orgJohn Ericsondid you enabled substitution on the builder too?20:02:55
@Ericson2314:matrix.orgJohn Ericsonthis seems not unlike the the issues that hydra.nixos.org was facing frankly20:03:12
@Ericson2314:matrix.orgJohn Ericsonthe thing I am working on right now (but it is a massive refactor) is making the builder not get dependent drvs so it that it has to subsitute or fail, and rebuilding things is not possible20:03:44
@Ericson2314:matrix.orgJohn Ericsonthat also cuts down on traffic between the queue ruunner and the builder20:03:54
@Ericson2314:matrix.orgJohn Ericsonthe current thing is how really old remote building worked20:04:09
@Ericson2314:matrix.orgJohn Ericsonthe thing I am going to is how hydra workedf or the last decade or so20:05:28
@frooastside:wolkeneis.devFrooastside I was not sure which setting is actually required because it didnt work after just enabling services.hydra-queue-builder-dev.useSubstitutes = true; (on the builder) so I also set nix.settings.builders-use-substitutes = true (I am not sure where i found that) and this is of course also set nix.settings.substituters = ["https://cache.nixos.org" "https://cache.MY CACHE"] 20:07:01
@Ericson2314:matrix.orgJohn Ericsonyeah taht sounds like it should work20:08:56
@Ericson2314:matrix.orgJohn Ericsonit is just hard to debug right now I am afraid with the dependency drvs being disclosed to the builder20:09:15
@Ericson2314:matrix.orgJohn Ericsonyou could try to check nix logs and figure out what it is doing what it is doing 20:09:28
@Ericson2314:matrix.orgJohn Ericsonbut that is hard20:09:31
@Ericson2314:matrix.orgJohn Ericson(the decision to build the stuff is nix's)20:10:28
@frooastside:wolkeneis.devFrooastside

This is the process running right now

hydra-q+   84985  0.0  0.2 595328 34872 ?        Sl   21:17   0:00 nix --extra-experimental-features nix-command build --json --no-pretty --print-build-logs --log-format raw-with-logs --no-link --max-silent-time 7200 --timeout 36000 --option max-build-log-size 67108864 --option fallback true --option substitute false --option builders  /nix/store/bx33y6dvsb44avnjah0hsh25hj0axa0f-users-groups.json.drv^*

So it picked the first drv that is not in the cache and tries to build it but --option substitute false is set

20:15:15
@frooastside:wolkeneis.devFrooastsideThats whats confusing me, how is it supposed to work? When the builder is missing dependencies, should nix get them because substitutions should be on, should the queue-builder get them from the cache or should the queue-runner send them to the builder?20:21:45
@Ericson2314:matrix.orgJohn Ericsoninteresting, I am not sure where that is coming from20:34:42

Show newer messages


Back to Room ListRoom Version: 6