!jtzdZrQSXtEpeGtzAn:nixos.org

Empty Room

58 Members
20 Servers

Load older messages


SenderMessageTime
27 Oct 2022
@hsngrmpf:matrix.orgDavHau joined the room.15:06:04
@roberthensing:matrix.orgRobert Hensing (roberth)
In reply to @whentze:matrix.org
does nixpkgs (the repo) implement the "no rocket science" rule btw?
generally, no. It is too large to test everything all the time. We do have some guided checks with passthru.tests or ofborg commands though
15:06:46
@whentze:matrix.orgWanja Hentzeis there a way to automatically deduce which tests could even theoretically be broken by a change?15:07:51
@whentze:matrix.orgWanja HentzeI feel like even `nix-build` almost gives that15:08:18
@roberthensing:matrix.orgRobert Hensing (roberth)
In reply to @whentze:matrix.org
would that be QA-for-nixpkgs or using-nix-for-QA? (or both)
I like this distinction. This room is mostly for the latter, but we should definitely have one for QA-for-nixpkgs too
15:08:24
@davidak:matrix.org@davidak:matrix.org
In reply to @whentze:matrix.org
is there a way to automatically deduce which tests could even theoretically be broken by a change?
nixpkgs-review builds reverse dependencies, but not their tests i think
15:08:54
@whentze:matrix.orgWanja Hentzewould it make sense to employ a "rollup" type workflow to reduce the cost of having all tests green always?15:09:42
@whentze:matrix.orgWanja Hentze(I have only a little experience with the nixpkgs repo but a bunch with rust-lang/rust so that's what I'm used to)15:10:27
@roberthensing:matrix.orgRobert Hensing (roberth)
In reply to @whentze:matrix.org
would it make sense to employ a "rollup" type workflow to reduce the cost of having all tests green always?
The staging workflow is kind of like this. A potential problem is that we basically never have an all-green situation. That's partly chick-and-egg, partly because Nixpkgs is huge.
15:13:01
@roberthensing:matrix.orgRobert Hensing (roberth)The automation would have to deal with false positives somehow and/or we'd have to fix all flaky tests15:13:53
@davidak:matrix.org@davidak:matrix.orgwe could mark tests as flaky, so they are ignored until the status is resolved. this is how the admin team i worked in handled false positives in our monitoring... few tests where ignored for years, because we don't had time to fix them15:18:49
@davidak:matrix.org@davidak:matrix.orgthe tests did still check every minute, but did not appear on the status monitor and did not send notigications. there was a specific list for them15:20:11
@davidak:matrix.org@davidak:matrix.org* the tests did still check every minute, but did not appear on the status monitor and did not send notifications. there was a specific list for them15:21:15
@nbp:mozilla.orgnbp If this is testing using Nix, one thing that is missing (or I am not aware that this was fixed) is that today, Nix needs a special set of environment variables to setup a testing environment.
(see https://github.com/NixOS/nixpkgs/blob/master/lib/tests/release.nix#L20-L34 )
This could be abstracted either with a command-line configuration file for all nix commands, or with a way a new nix tests --init, nix tests run <command running nix tests> or which output the set of environment variables to be used.
15:22:02
@roberthensing:matrix.orgRobert Hensing (roberth)I feel like the always green CI issue has been discussed many times before, but I couldn't find an issue on nixpkgs. A more persistent medium like that may be more productive for this; gather knowledge, coordinating experiments and development.15:26:22
@0x4a6f:matrix.org[0x4A6F] joined the room.15:27:04
@roberthensing:matrix.orgRobert Hensing (roberth) nbp: seems like a valid issue to raise in the nix project 15:28:25
@roberthensing:matrix.orgRobert Hensing (roberth) I don't know if nix tests --init is the right solution, partly because environment variables require a little dance to load into the calling shell 15:29:40
@roberthensing:matrix.orgRobert Hensing (roberth)running Nix inside the sandbox needs better support anyway because of RFC 92, computed derivations15:30:54
@danielbarter:matrix.orgdanielbarter joined the room.15:34:22
@profpatsch:augsburg.oneProfpatsch joined the room.15:42:53
@alejandrosame:matrix.orgalejandrosame joined the room.16:39:43
@infinisil:matrix.orginfinisil joined the room.16:47:21
@jtojnar:matrix.orgJan Tojnar joined the room.17:50:19
@pinage404:matrix.orgpinage404 joined the room.21:50:04
28 Oct 2022
@denna:matrix.orgolaf joined the room.16:21:54
9 Nov 2022
@roberthensing:matrix.orgRobert Hensing (roberth) Would someone like to review testBuildFailure and testEqualContents? https://github.com/NixOS/nixpkgs/pull/196251 12:56:02
16 Nov 2022
@omlet:matrix.orgomlet joined the room.20:34:05
18 Nov 2022
@kraftnix:matrix.orgkraftnix joined the room.17:24:41
26 Nov 2022
@shane:sveller.socialshanesveller joined the room.19:22:50

There are no newer messages yet.


Back to Room ListRoom Version: 9