| 27 Oct 2022 |
 @davidak:matrix.org | we could mark tests as flaky, so they are ignored until the status is resolved. this is how the admin team i worked in handled false positives in our monitoring... few tests where ignored for years, because we don't had time to fix them | 15:18:49 |
| @davidak:matrix.org | the tests did still check every minute, but did not appear on the status monitor and did not send notigications. there was a specific list for them | 15:20:11 |
| @davidak:matrix.org | * the tests did still check every minute, but did not appear on the status monitor and did not send notifications. there was a specific list for them | 15:21:15 |
 nbp | If this is testing using Nix, one thing that is missing (or I am not aware that this was fixed) is that today, Nix needs a special set of environment variables to setup a testing environment.
(see https://github.com/NixOS/nixpkgs/blob/master/lib/tests/release.nix#L20-L34 )
This could be abstracted either with a command-line configuration file for all nix commands, or with a way a new nix tests --init, nix tests run <command running nix tests> or which output the set of environment variables to be used. | 15:22:02 |
 Robert Hensing (roberth) | I feel like the always green CI issue has been discussed many times before, but I couldn't find an issue on nixpkgs. A more persistent medium like that may be more productive for this; gather knowledge, coordinating experiments and development. | 15:26:22 |
