| 21 Jun 2022 |
 ncfavier | looks like https://github.com/NixOS/ofborg/issues/504 | 21:18:30 |
| 22 Jun 2022 |
 hexa | indeed, ofborg seems to be behaving weirdly for tests 😕 | 14:34:21 |
| hexa | many queued jobs, all unfinished | 14:35:50 |
| 25 Jun 2022 |
 Sandro 🐧 | ofborg logs are currently not loading for me | 22:25:59 |
| Sandro 🐧 | the websocket is only sending empty 1 bytes messages after the subscribe | 22:26:46 |
| 27 Jun 2022 |
|  underpantsgnome joined the room. | 08:21:13 |
| 28 Jun 2022 |
|  ma27 changed their profile picture. | 11:38:05 |
| 29 Jun 2022 |
| Sandro 🐧 | How did it happen that we lose aarch64-darwin? | 12:28:04 |
 Alyssa Ross | hmm, OfBorg doesn't seem to be noticing my @OfBorg build comments here https://github.com/NixOS/nixpkgs/pull/179556#issuecomment-1170407607 | 20:17:29 |
| 7 Jul 2022 |
 Winter (she/her) | Is there a reason that failures during test eval are reported as neutral and not failing? | 04:17:39 |
 7c6f434c | Yes. | 04:18:02 |
| 7c6f434c | There are flaky tests, failed-before-anyway, disk full issues, timeouts… | 04:18:49 |
| 7c6f434c | (also complicated issues with some subset of the builders…) | 04:19:26 |
| 7c6f434c | Equating this to eval failures dilutes the clear «oops» status of reported definite-failures | 04:20:31 |
| Winter (she/her) | In reply to @7c6f434c:nitro.chat
Equating this to eval failures dilutes the clear «oops» status of reported definite-failures Wait, just to be clear: actual failures during a test run are reported as failures, just not eval errors? Or are they all reported as neutral? | 04:22:55 |
| 7c6f434c | evail failures are failures
| 04:23:20 |
| 7c6f434c | build/test failures are neutral | 04:23:28 |
| Winter (she/her) | ah, okay, thank you | 04:38:00 |
| Winter (she/her) | build/test failures i understand, but why can't eval failures be reported as failures? those shouldn't be problematic even in weird machines (i'd think...)
sorry for asking basically the same thing again | 04:39:19 |
| 7c6f434c | Eval failures are failures, yes | 04:39:49 |
| Winter (she/her) | test eval failures, though? | 05:49:30 |
| Winter (she/her) | like, when evaling a test(s) | 05:49:46 |
| 7c6f434c | Well, technically speaking pkg.tests not being there is a) eval failure b) definitely not a check failure… | 05:50:47 |
| Winter (she/her) | I mean when manually triggering a test and it's eval failing, that's reported as neutral | 05:51:21 |
| 7c6f434c | For example, a test missing would be an eval failure! | 05:51:44 |
| 7c6f434c | And possibly just an invocation typo | 05:51:58 |
| Winter (she/her) | Right, but that's reported as a neutral failure to GH. | 05:53:01 |
| 7c6f434c | As it should be | 05:53:13 |
| 7c6f434c | And also some tests might be assuming a platform and be useful as such and have eval failures on unexpected platforms | 05:53:58 |
| Winter (she/her) | I guess I'm confining my thinking to NixOS tests when we have different types of tests in-tree as well. | 05:54:38 |
