How does a user specify input, precisely? If I look for example at the Rust doctests page, there's a fair bit there about syntax. There's also the additional constraint that these things exist in doc strings. We don't really/necessarily have docstrings here, so how do I present a list of assertions for a nix repl to check?

What kind of output does does a doc tester produce (is it just a pass/fail, is there a way to extract the LHS and RHS of the input assertions)? Would you find it useful to co-opt the language/structure in the issue linked above, or does it pose unnecessary constraints on your work?

In reply to @mightyiam:matrix.org
I lost you at "input". What's input, please?

In reply to @mightyiam:matrix.org
I lost you at "input". What's input, please?

How does a user specify input, precisely? If I look for example at the Rust doctests page, there's a fair bit there about syntax. There's also the additional constraint that these things exist in doc strings. We don't really/necessarily have docstrings here, so how do I present to the program what is essentially a list of assertions for a nix repl to check?

What kind of output does does a doc tester produce (is it just a pass/fail, is there a way to extract the LHS and RHS of the input assertions)? Would you find it useful to co-opt the language/structure in the issue linked above, or does it pose unnecessary constraints on your work?

In reply to @mightyiam:matrix.org
We're working on this in mob format regularly. Wanna join?

In reply to @bzzm3r:matrix.org

Based on your input: https://github.com/NixOS/nix.dev/issues/864

I am not sure if this qualifies as low-tech. Maybe it does because all it uses is a CLI tool. With modern tools, it is easy to make a heavily automatically documented CLI tool (e.g. if using Rust: can use bpaf/clap both convert doc strings into CLI help strings).

However, low-tech ought to be measured against cost-of-manual-work? In this particular case: cost of copying+pasting, verifying output expression against expectation, validating that examples remain valid across Nix updates.

Low-tech is a really bad term to describe what I mean, sorry. It’s really about three things:

1. Lean towards manipulating data rather than code that does things (cf. "show me your tables“, declarative programming)
2. Minimize length of processing chain and number of indirections (less stuff to understand)
3. Prefer widespread, well-understood tools, even if they’re somewhat wrong (less new stuff to learn)

And of course it’s a trade-off between amount of ongoing manual work and threshold to contribute. Because the system is so large and the number of people with deep knowledge so small, I’m inclined to lower the threshold and hope to attract more talent with a gentle learning curve. The more people care, the more will eventually run into the hard problems, the more may have the right combination of time, skill, and motivation to finally solve them.