| 16 May 2024 |
 Qyriad | it's pitiful | 17:24:10 |
 samrose | heh | 17:24:16 |
 raitobezarius | expanding it is cool | 17:24:23 |
| raitobezarius | writing new tests for builtins which are not tested | 17:24:30 |
| raitobezarius | new test behaviors, etc. | 17:24:32 |
| Qyriad | we have three flavors of test:
gtest (offer only available in libexpr and libutil) bash script virtual machine
the vast, vast majority of testing is in the "bash script" flavor and it is a mess | 17:25:01 |
| samrose | I was just going to ask on the "functional" tests: do we still like using bash there? | 17:25:56 |
| samrose | the last time that I worked on a major nix related cli project that used bash, or bats for testing, over time it became rather kind of hard to maintain | 17:26:42 |
| samrose | I am not usually a big python fan, but in that project we heard from some in the Rust community that they actually use Python to test CLI and seem to have success there. | 17:27:59 |
| Qyriad | no gods please kill bash testing. the problem is that it's kind of really difficult to migrate an entire test suite and be sure that you actually migrated the test suite correctly and won't lose coverage accidentally in the process, which makes any kind of migration a bit nerve wracking | 17:28:36 |
| samrose | yes it's a rather large undertaking | 17:28:52 |
| samrose | could be done in chunks | 17:28:59 |
| Qyriad | at the very least though, it would be wonderful to have an option for new tests to not just be a mess of bash | 17:29:17 |
| samrose | yes it could be to choose a new approach that this community is truly happy with, rewrite some of the old tests, use it for all new, and then over time rewrite all old | 17:30:00 |
| Qyriad | yeah | 17:30:31 |
| samrose | and also probably you all would want to speak into the testing paradigm to make sure the tests are relevant | 17:30:52 |
| Qyriad | mhmm | 17:33:22 |
| Qyriad | In reply to@samrose:matrix.org
I am not usually a big python fan, but in that project we heard from some in the Rust community that they actually use Python to test CLI and seem to have success there. Python's fine, and it's already part of the buildsystem in other ways. If there's something that's a better idea or more preferred for another reason we can talk about that too | 17:35:04 |
| samrose | In reply to @qyriad:katesiria.org
Python's fine, and it's already part of the buildsystem in other ways. If there's something that's a better idea or more preferred for another reason we can talk about that too It would be nice if someone had written a general cli testing suit in Rust, or Go (which likely would make the test suite run faster) but I am not aware of anything that exists. I think the Rust cli tool community gravitated toward using python due to Pexpect, and a lot of existing tools that are good fit for test suite. | 17:37:19 |
| samrose | (it seems like functional tests need the most help basically if I am not mistaken. Apologies if you basically already said this) | 17:38:19 |
| Qyriad | no they definitely do. those are the "bash" flavor of tests | 17:38:40 |
| samrose | https://git.lix.systems/lix-project/lix/src/branch/main/tests/functional | 17:38:51 |
| Qyriad | yep | 17:38:57 |
| Qyriad | something that probably could be migrated separately is the stuff that sets up the daemon the test suite uses | 17:39:16 |
|  aprl joined the room. | 18:30:38 |
 puck | domen?? https://github.com/NixOS/nixpkgs/blob/741342772c768588878be460f0e1aa9e6d8d0caa/pkgs/by-name/de/devenv/package.nix#L19-L30 | 18:45:40 |
| Qyriad | o.o | 18:46:06 |
| puck | pushed a fix lmao | 18:51:40 |
| Qyriad | Wait to what | 18:52:17 |
| puck | nixos-module; devenv wouldn't build with the overlay attached because it overrided the source of pkgs.nix | 18:52:41 |
