| 27 Oct 2022 |
 Wanja Hentze | would it make sense to employ a "rollup" type workflow to reduce the cost of having all tests green always? | 15:09:42 |
| Wanja Hentze | (I have only a little experience with the nixpkgs repo but a bunch with rust-lang/rust so that's what I'm used to) | 15:10:27 |
 Robert Hensing (roberth) | In reply to @whentze:matrix.org
would it make sense to employ a "rollup" type workflow to reduce the cost of having all tests green always? The staging workflow is kind of like this. A potential problem is that we basically never have an all-green situation. That's partly chick-and-egg, partly because Nixpkgs is huge. | 15:13:01 |
| Robert Hensing (roberth) | The automation would have to deal with false positives somehow and/or we'd have to fix all flaky tests | 15:13:53 |
 @davidak:matrix.org | we could mark tests as flaky, so they are ignored until the status is resolved. this is how the admin team i worked in handled false positives in our monitoring... few tests where ignored for years, because we don't had time to fix them | 15:18:49 |
| @davidak:matrix.org | the tests did still check every minute, but did not appear on the status monitor and did not send notigications. there was a specific list for them | 15:20:11 |
| @davidak:matrix.org | * the tests did still check every minute, but did not appear on the status monitor and did not send notifications. there was a specific list for them | 15:21:15 |
 nbp | If this is testing using Nix, one thing that is missing (or I am not aware that this was fixed) is that today, Nix needs a special set of environment variables to setup a testing environment.
(see https://github.com/NixOS/nixpkgs/blob/master/lib/tests/release.nix#L20-L34 )
This could be abstracted either with a command-line configuration file for all nix commands, or with a way a new nix tests --init, nix tests run <command running nix tests> or which output the set of environment variables to be used. | 15:22:02 |
| Robert Hensing (roberth) | I feel like the always green CI issue has been discussed many times before, but I couldn't find an issue on nixpkgs. A more persistent medium like that may be more productive for this; gather knowledge, coordinating experiments and development. | 15:26:22 |
|  [0x4A6F] joined the room. | 15:27:04 |
| Robert Hensing (roberth) | nbp: seems like a valid issue to raise in the nix project | 15:28:25 |
| Robert Hensing (roberth) | I don't know if nix tests --init is the right solution, partly because environment variables require a little dance to load into the calling shell | 15:29:40 |
| Robert Hensing (roberth) | running Nix inside the sandbox needs better support anyway because of RFC 92, computed derivations | 15:30:54 |
|  danielbarter joined the room. | 15:34:22 |
|  Profpatsch joined the room. | 15:42:53 |
|  alejandrosame joined the room. | 16:39:43 |
|  infinisil joined the room. | 16:47:21 |
|  Jan Tojnar joined the room. | 17:50:19 |
|  pinage404 joined the room. | 21:50:04 |
| 28 Oct 2022 |
|  olaf joined the room. | 16:21:54 |
| 9 Nov 2022 |
| Robert Hensing (roberth) | Would someone like to review testBuildFailure and testEqualContents? https://github.com/NixOS/nixpkgs/pull/196251 | 12:56:02 |
| 16 Nov 2022 |
|  omlet joined the room. | 20:34:05 |
| 18 Nov 2022 |
|  kraftnix joined the room. | 17:24:41 |
| 26 Nov 2022 |
|  shanesveller joined the room. | 19:22:50 |
| 29 Nov 2022 |
 raitobezarius | I pushed on 8001336320fba24158a0474e574e5ede1543a6ba to test https://github.com/NixOS/nixpkgs/pull/203641 | 18:35:57 |
| raitobezarius | on python-test-refactoring for Hydra evaluationb | 18:36:06 |
| raitobezarius | * on python-test-refactoring for Hydra evaluation | 18:36:07 |
| raitobezarius | FWIW | 18:36:08 |
| 1 Dec 2022 |
| omlet left the room. | 16:27:25 |
| 2 Jan 2023 |
| nbp changed their display name from nbp to nbp | {pto until 2023-01-11}. | 09:46:29 |
