!RROtHmAaQIkiJzJZZE:nixos.org

NixOS Infrastructure

349 Members
Next Infra call: 2024-07-11, 18:00 CEST (UTC+2) | Infra operational issues backlog: https://github.com/orgs/NixOS/projects/52 | See #infra-alerts:nixos.org for real time alerts from Prometheus.108 Servers

Load older messages


SenderMessageTime
26 Jun 2025
@ctheune:matrix.flyingcircus.ioChristian Theune well. a colleague pointed out that i can leverage a local repo's working dir with the builtin fetchgit, so that' fast now. still a 25 min cycle and about 13 bisect steps left (with the added complication that there is another v9fs bug in the suspected range that I'll have to skip to see where the bug that is biting us was introduced ... ) 19:45:38
@ctheune:matrix.flyingcircus.ioChristian Theuneso i can't even automate that properly because i need to manually inspect each test failure that always deviates a bit. i guess i need an AI agent /s19:46:20
@raitobezarius:matrix.orgraitobezariusGiven what you are saying, I believe the Lix project already bisected the kernel bug you are looking for, unfortunately, all the commit IDs and answers are trapped behind the kernel lore mailing list browser: https://lore.kernel.org/all/w5ap2zcsatkx4dmakrkjmaexwh3mnmgc5vhavb2miaj6grrzat@7kzr5vlsrmh5/19:51:38
@hexa:lossy.networkhexa (signing key rotation when) Christian Theune: ^ 19:51:57
@ctheune:matrix.flyingcircus.ioChristian Theuneoh?19:52:43
* @ctheune:matrix.flyingcircus.ioChristian Theune checks19:52:46
@ctheune:matrix.flyingcircus.ioChristian Theunefor me it's behind a guru error 😉19:53:10
@raitobezarius:matrix.orgraitobezariusI think you will get a 503 unfortunately19:53:10
@ctheune:matrix.flyingcircus.ioChristian Theuneyeah19:53:14
@ctheune:matrix.flyingcircus.ioChristian Theunedid this break recently / due to a current event? i've been working with it last week I think ... 19:53:45
@raitobezarius:matrix.orgraitobezariushttps://lkml.indiana.edu/hypermail/linux/kernel/2506.1/07850.html19:54:24
@raitobezarius:matrix.orgraitobezariusthankfully I have found another server19:54:31
@raitobezarius:matrix.orgraitobezariusTo me, this is not a regression, this was always broken, it just takes a lot of I/O in the VM to materialize the regression19:54:50
@raitobezarius:matrix.orgraitobezariusYou can try to revert the "real fix", apply the workaround and verify if this fixes your issues19:55:07
@raitobezarius:matrix.orgraitobezariusIf it does, this is the folio management of 9pfs which is still broken atm19:55:15
@ctheune:matrix.flyingcircus.ioChristian Theuneugh 19:56:04
@ctheune:matrix.flyingcircus.ioChristian Theuneso yes19:56:07
@ctheune:matrix.flyingcircus.ioChristian Theunein your mail, the last point: "massive io in the guest" is what is triggering in our cases, too19:56:22
@ctheune:matrix.flyingcircus.ioChristian Theuneand you mentioning folios makes me weep19:56:33
@ctheune:matrix.flyingcircus.ioChristian Theunei already had a 3 year journey wrapped up of getting an xfs folio issue fixed last year -_-19:56:51
@raitobezarius:matrix.orgraitobezarius Well, you will be happy to know that the folio code is going to happen even more in more FS subsystems 19:57:12
@qyliss:fairydust.spaceAlyssa RossDidn't we switch to virtio-fs for VM tests?19:57:24
@ctheune:matrix.flyingcircus.ioChristian Theunelinus was not amused when he saw what was going on ... 19:57:31
@ctheune:matrix.flyingcircus.ioChristian Theune Alyssa Ross: 24.11 and 25.05 don't seem to be, might it have been more recent? 19:58:15
@ctheune:matrix.flyingcircus.ioChristian Theune raitobezarius: do you happen to have a patch set for 6.12.34 around that I can try out? 19:58:51
@qyliss:fairydust.spaceAlyssa RossNot more recent than 25.0519:59:13
@qyliss:fairydust.spaceAlyssa RossAh it was buildInLinuxVM, not NixOS tests19:59:38
@ctheune:matrix.flyingcircus.ioChristian Theunehmm. now ... how can i respond to that regression list thread so I get involved there ... o_O20:01:56
@qyliss:fairydust.spaceAlyssa RossDownload the raw message, open in it your mail client20:03:10
@qyliss:fairydust.spaceAlyssa RossAlthough you might need lore to come back for that20:03:32

Show newer messages


Back to Room ListRoom Version: 6