| 19 Oct 2021 |
 K900 | I've never touched nixos infra before | 12:26:37 |
 Janne Heß | we've seen that as well, restarting might help | 12:27:34 |
| Janne Heß | Maybe fixing that will also fix the test getting randomly stuck sometimes 🤔 | 12:31:21 |
 @grahamc:nixos.org | what is the issue in that test? | 13:25:57 |
| Janne Heß | sorry, the testss | 13:28:09 |
| Janne Heß | * sorry, the tests | 13:28:15 |
| Janne Heß | looks like (at least for us) they sometimes get stuck while reading the output of a command:
$ sudo py-spy dump --pid 3774992
Process 3774992: /nix/store/97w52ckcjnfiz89h3lh7zf1kysgfm2s8-python3-3.9.6/bin/python3 /nix/store/pnfqk9c4w65jwyp0fcskawp404r2lvim-nixos-test-driver/bin/.nixos-test-driver-wrapped
Python v3.9.6 (/nix/store/97w52ckcjnfiz89h3lh7zf1kysgfm2s8-python3-3.9.6/bin/python3.9)
Thread 0x7FFFF77EB340 (active)
execute (.nixos-test-driver-wrapped:586)
systemctl (.nixos-test-driver-wrapped:561)
get_unit_info (.nixos-test-driver-wrapped:530)
check_active (.nixos-test-driver-wrapped:509)
retry (.nixos-test-driver-wrapped:193)
wait_for_unit (.nixos-test-driver-wrapped:527)
<module> (<string>:118)
test_script (.nixos-test-driver-wrapped:1175)
run_tests (.nixos-test-driver-wrapped:1179)
<module> (.nixos-test-driver-wrapped:1304)
Thread 0x7FFFF61D1640 (active)
process_serial_output (.nixos-test-driver-wrapped:918)
| 13:28:47 |
| K900 | It won't :( | 13:31:02 |
| K900 | It doesn't even touch that code | 13:31:06 |
| K900 | Oh no | 13:33:16 |
| K900 | I've looked at how execute is implemented | 13:33:23 |
| Janne Heß | I did that too and then went on doing anything to unsee that | 13:34:03 |
| K900 | i will not attempt to refactor the nixos test infra i will not attempt to refactor the nixos test infra i will not attempt to | 13:34:36 |
| Janne Heß | I have also said that about switch-to-configuration when I was younger… it was a lot of pain but it paid off. So go for it 🙃 | 13:36:20 |
 nf | In reply to @k900:0upti.me
I've looked at how execute is implemented seems like it will fail to detect the magic string if it's split across chunks | 16:53:23 |
| nf | that could cause things to hang that weren't supposed to | 16:53:39 |
| K900 | Or if the program ever outputs the magic string | 16:54:15 |
| K900 | Even by accident | 16:54:24 |
| nf | yeah | 16:54:34 |
| nf | though that seems less likely | 16:54:58 |
| 20 Oct 2021 |
 Domen Kožar | Threading code in python test driver is iffy | 11:49:58 |
 hexa | The atexit handler together with the threading seems problematic | 16:39:01 |
| K900 | It's fine actually | 16:41:20 |
| K900 | atexit handlers run before threads are destroyed | 16:41:37 |
| K900 | (which is how threading cleans up after itself, among other things) | 16:41:44 |
| K900 | https://hydra.nixos.org/build/156403450/nixlog/298 it's broken again :( | 18:24:24 |
| hexa |
Fatal Python error: _enter_buffered_busy: could not acquire lock for <_io.BufferedWriter name='<stderr>'> at interpreter shutdown, possibly due to daemon threads
| 18:37:59 |
| hexa | logging thread likely wants to print, but the writer is closed due to thread shutdown | 18:38:17 |
| hexa | maybe the last commit that touched that bit should be reverted | 18:38:36 |
| hexa | failing tests that don't relate to the actual tests are annoying | 18:38:45 |
