| 26 Jun 2025 |
 hexa | fscking for the thrill of it | 19:06:06 |
| hexa | follow me for more silly infra stories 🔔 | 19:06:40 |
| hexa | nope, will try git fetch --prune first 😄 | 19:09:07 |
| hexa | Resolving deltas: 100% (133768/133768), completed with 9831 local objects.
fatal: bad object 4fd048a763a7fd02f20f0c59918e24fab3e19215
error: https://github.com/NixOS/nixpkgs.git did not send all necessary objects
Auto packing the repository in background for optimum performance.
See "git help gc" for manual housekeeping.
| 19:19:57 |
| hexa | will be swapping out the repo, everything else is madness 😄 | 19:34:09 |
 Christian Theune | I feel you. i'm currently bisecting the 6.12 series for another v9fs bug that is biting us in our nixos tests ... i started out with getting rate limited by the kernel.org git server because I cloned the repo over and over ... 🙈 ... took a bit to see what's going on, but strace showed me that it was throttled at 1 object per second ... which is ... a tarpit I guess ... | 19:43:46 |
| hexa | that sounds bad, yeah 😄 | 19:44:15 |
| Christian Theune | well. a colleague pointed out that i can leverage a local repo's working dir with the builtin fetchgit, so that' fast now. still a 25 min cycle and about 13 bisect steps left (with the added complication that there is another v9fs bug in the suspected range that I'll have to skip to see where the bug that is biting us was introduced ... ) | 19:45:38 |
| Christian Theune | so i can't even automate that properly because i need to manually inspect each test failure that always deviates a bit. i guess i need an AI agent /s | 19:46:20 |
 raitobezarius | Given what you are saying, I believe the Lix project already bisected the kernel bug you are looking for, unfortunately, all the commit IDs and answers are trapped behind the kernel lore mailing list browser: https://lore.kernel.org/all/w5ap2zcsatkx4dmakrkjmaexwh3mnmgc5vhavb2miaj6grrzat@7kzr5vlsrmh5/ | 19:51:38 |
| hexa | Christian Theune: ^ | 19:51:57 |
| Christian Theune | oh? | 19:52:43 |
| * Christian Theune checks | 19:52:46 |
| Christian Theune | for me it's behind a guru error 😉 | 19:53:10 |
| raitobezarius | I think you will get a 503 unfortunately | 19:53:10 |
| Christian Theune | yeah | 19:53:14 |
| Christian Theune | did this break recently / due to a current event? i've been working with it last week I think ... | 19:53:45 |
| raitobezarius | https://lkml.indiana.edu/hypermail/linux/kernel/2506.1/07850.html | 19:54:24 |
| raitobezarius | thankfully I have found another server | 19:54:31 |
| raitobezarius | To me, this is not a regression, this was always broken, it just takes a lot of I/O in the VM to materialize the regression | 19:54:50 |
| raitobezarius | You can try to revert the "real fix", apply the workaround and verify if this fixes your issues | 19:55:07 |
| raitobezarius | If it does, this is the folio management of 9pfs which is still broken atm | 19:55:15 |
| Christian Theune | ugh | 19:56:04 |
| Christian Theune | so yes | 19:56:07 |
| Christian Theune | in your mail, the last point: "massive io in the guest" is what is triggering in our cases, too | 19:56:22 |
| Christian Theune | and you mentioning folios makes me weep | 19:56:33 |
| Christian Theune | i already had a 3 year journey wrapped up of getting an xfs folio issue fixed last year -_- | 19:56:51 |
| raitobezarius | Well, you will be happy to know that the folio code is going to happen even more in more FS subsystems | 19:57:12 |
 Alyssa Ross | Didn't we switch to virtio-fs for VM tests? | 19:57:24 |
| Christian Theune | linus was not amused when he saw what was going on ... | 19:57:31 |
