fricklerhandwerk: Hello again. Sorry for my delay in responding, I'm just coming off a rather busy few days, but I had a pending to write a response to all the interesting things you mentioned!

Especially the official documentation should be accessible for anyone. It's just that we currently focus more on developer needs and use cases simply due to limited resources to set a high bar of quality. Also NixOS does not get much love, as far as I know, there is currently no one who is deeply engaged with the NixOS manual.

This is how it should be, but I believe that at the moment it is not; and because of the aspirations about the documentation I find it hard to believe that without changes in the way of thinking its approach, it will become accessible and familiar to a new user even in the medium term (years).

We do have the non-linear (reference-style) presentation for packages and NixOS modules you're referring to on search.nixos.org: Here's an example for zathura. Changing something about that is technically much more involved than just writing stuff into a wiki – you'll either have to get involved in developing the search site, or the packages themselves. Still, that is the correct way to do it, because it will fix problems for all packages and for everyone at once. I had very rewarding encounters with the maintainers.

(Thank you very much for the link, I didn't know those details about documentation styles!) That's not really what I'm referring to. Certainly that corresponds to non-linear documentation, but it's not comprehensive. I remember running into the question, before I really got into the Nix environment, why there are so many ways to install, what each is for, and what potential uses they might have. That information is not easy to add to search.nixos.org, particularly when you want to document peculiarities about a specific package.

I understand that a project of this size requires having mechanisms to automate and simultaneously address a large number of packages, however I do not agree that this is the right way. Personally I consider it more practical to break down barriers to create documentation if it allows more people to do it and it is less dispersed. In that sense the documentation offered by search.nixos.org should be a complement to the documentation, not the documentation itself.

(By the way, if you have the opportunity, I would very much like to hear the conclusions of your meetings with the search.nixos.org maintainers!)

Right now, as Yuki mentions, override options are not displayed, but that is a deeper technical issue due to how the Nix language works and how Nixpkgs is organised.

Could you explain a little more about this technical problem? I don't think there is an automatable way to provide information about this. Those who can give useful information about how to use a package properly through its relationship with others or its construction arguments are people. However, since we may be talking about different things, I'll give you an example of what I meant. Recently I wanted to add support for some additional extensions to my image viewer "feh". After several days spent, I understood what I had to do. And after one more afternoon, I understood that I had to overwrite a library of his inputs for another package with extended capabilities. I would LOVE to have an official place to document that situation, and save someone else that time and, perhaps, take some of the friction out of their experience with Nix.

That is also a meta-problem that I wish we could solve step by step. The Wiki articles you posted are prime examples of what should be in the official documentation. Flakes should be much better documented, and that documentation much easier to find in the Nix manual. The Python development workflow should be covered on nix.dev, and the relevant details elaborated on in the Nixpkgs manual. That the Wiki is preferred by search engines over manuals is also an issue that has to be addressed from different directions.

But, don't you think the problem is perhaps the approach? (I will make some comments below that I may be completely wrong about, and that I imagine you addressed at some point. If so, I would appreciate it if you would correct me and refer me to where I can find the respective discussions.) There are differences between the terms ordering and organizing. Ordering is simply assigning a pattern. Organizing REQUIRES that pattern to facilitate access to that which is being ordered. My perception of the Nix documentation is that it is very well ordered through manuals and additional resources, but not very well organized. If I as a user wanted to understand how Python works in the Nix ecosystem I would want to have ONE source that would contextualize and show me, from a general and deep overview, how I should understand and work with it (even if that requires me to go to other sources, that's fine). If on the contrary to understand it I have to open Nix.dev, nixpkgs manual, nix manual, nixos manual, nixos manual, home-manager options, 3 Nix Discourse threads, and 4 reddit questions, the only thing I can conclude at a first glance is a disjointed and poorly integrated environment for which I have to read, learn and understand a lot of things that in a distribution like Arch, the wiki documents it for me in the same page. I don't consider that fragmentation of information is the right direction, but on the contrary it should be to unify it, integrate it and direct it in ways that come naturally to people.

That is fine, and dropping information on edge cases is exactly what the Wiki is suitable for. The more general case "here's all the regular things you can do with the package" should be automatically provided from the source code. The Wiki cannot possibly be synchronised with the source, so any attempt is doomed to go out of date and confuse people even more.

I am aware of the sad history that the wiki has had at Nix since its inception, and that perhaps that is where the care in making the code provide the documentation comes from; but it cannot abstract as much information from the code as the active use of the users can convey, nor does it allow as much flexibility as documentation created by the community for the community.

So by all means, please add to the Wiki as you see fit. I only strongly discourage duplicating information that is already in the manuals or in other sources that are derived directly from the code.

I will try to make my first contributions in the following days, and I will try to follow your indications as much as possible!

To provide some more context on this, in terms of documentation I see the ecosystem stuck in an uncanny valley: Too much information is only available in the source, the good stuff that's human-readable is hard to find, and unofficial, uncurated resources such as the Wiki or blog posts in varying degrees of correctness and up-to-date-ness appear front and center on search engines. Changing any of that involves lots of detail knowledge and attention from maintainers...

I totally agree on your perspective, but I feel like we could do a lot more integrating the content than fragmenting it. And I'd really like to hear from you or someone else on the team about what opinions they have formed regarding this fact. Any references you can give me on discussions in this regard (or also Nix documentation in general) I'd love to read them.

Again, thanks for reaching out. Feel free to ask for help if you like to get involved!

Thanks to you for your kind and extensive response! Happy New Year!

In reply to @wiseh:matrix.org

fricklerhandwerk: Hello again. Sorry for my delay in responding, I'm just coming off a rather busy few days, but I had a pending to write a response to all the interesting things you mentioned!

Especially the official documentation should be accessible for anyone. It's just that we currently focus more on developer needs and use cases simply due to limited resources to set a high bar of quality. Also NixOS does not get much love, as far as I know, there is currently no one who is deeply engaged with the NixOS manual.

This is how it should be, but I believe that at the moment it is not; and because of the aspirations about the documentation I find it hard to believe that without changes in the way of thinking its approach, it will become accessible and familiar to a new user even in the medium term (years).

We do have the non-linear (reference-style) presentation for packages and NixOS modules you're referring to on search.nixos.org: Here's an example for zathura. Changing something about that is technically much more involved than just writing stuff into a wiki – you'll either have to get involved in developing the search site, or the packages themselves. Still, that is the correct way to do it, because it will fix problems for all packages and for everyone at once. I had very rewarding encounters with the maintainers.

(Thank you very much for the link, I didn't know those details about documentation styles!) That's not really what I'm referring to. Certainly that corresponds to non-linear documentation, but it's not comprehensive. I remember running into the question, before I really got into the Nix environment, why there are so many ways to install, what each is for, and what potential uses they might have. That information is not easy to add to search.nixos.org, particularly when you want to document peculiarities about a specific package.

I understand that a project of this size requires having mechanisms to automate and simultaneously address a large number of packages, however I do not agree that this is the right way. Personally I consider it more practical to break down barriers to create documentation if it allows more people to do it and it is less dispersed. In that sense the documentation offered by search.nixos.org should be a complement to the documentation, not the documentation itself.

(By the way, if you have the opportunity, I would very much like to hear the conclusions of your meetings with the search.nixos.org maintainers!)

Right now, as Yuki mentions, override options are not displayed, but that is a deeper technical issue due to how the Nix language works and how Nixpkgs is organised.

Could you explain a little more about this technical problem? I don't think there is an automatable way to provide information about this. Those who can give useful information about how to use a package properly through its relationship with others or its construction arguments are people. However, since we may be talking about different things, I'll give you an example of what I meant. Recently I wanted to add support for some additional extensions to my image viewer "feh". After several days spent, I understood what I had to do. And after one more afternoon, I understood that I had to overwrite a library of his inputs for another package with extended capabilities. I would LOVE to have an official place to document that situation, and save someone else that time and, perhaps, take some of the friction out of their experience with Nix.

That is also a meta-problem that I wish we could solve step by step. The Wiki articles you posted are prime examples of what should be in the official documentation. Flakes should be much better documented, and that documentation much easier to find in the Nix manual. The Python development workflow should be covered on nix.dev, and the relevant details elaborated on in the Nixpkgs manual. That the Wiki is preferred by search engines over manuals is also an issue that has to be addressed from different directions.

But, don't you think the problem is perhaps the approach? (I will make some comments below that I may be completely wrong about, and that I imagine you addressed at some point. If so, I would appreciate it if you would correct me and refer me to where I can find the respective discussions.) There are differences between the terms ordering and organizing. Ordering is simply assigning a pattern. Organizing REQUIRES that pattern to facilitate access to that which is being ordered. My perception of the Nix documentation is that it is very well ordered through manuals and additional resources, but not very well organized. If I as a user wanted to understand how Python works in the Nix ecosystem I would want to have ONE source that would contextualize and show me, from a general and deep overview, how I should understand and work with it (even if that requires me to go to other sources, that's fine). If on the contrary to understand it I have to open Nix.dev, nixpkgs manual, nix manual, nixos manual, nixos manual, home-manager options, 3 Nix Discourse threads, and 4 reddit questions, the only thing I can conclude at a first glance is a disjointed and poorly integrated environment for which I have to read, learn and understand a lot of things that in a distribution like Arch, the wiki documents it for me in the same page. I don't consider that fragmentation of information is the right direction, but on the contrary it should be to unify it, integrate it and direct it in ways that come naturally to people.

That is fine, and dropping information on edge cases is exactly what the Wiki is suitable for. The more general case "here's all the regular things you can do with the package" should be automatically provided from the source code. The Wiki cannot possibly be synchronised with the source, so any attempt is doomed to go out of date and confuse people even more.

I am aware of the sad history that the wiki has had at Nix since its inception, and that perhaps that is where the care in making the code provide the documentation comes from; but it cannot abstract as much information from the code as the active use of the users can convey, nor does it allow as much flexibility as documentation created by the community for the community.

So by all means, please add to the Wiki as you see fit. I only strongly discourage duplicating information that is already in the manuals or in other sources that are derived directly from the code.

I will try to make my first contributions in the following days, and I will try to follow your indications as much as possible!

To provide some more context on this, in terms of documentation I see the ecosystem stuck in an uncanny valley: Too much information is only available in the source, the good stuff that's human-readable is hard to find, and unofficial, uncurated resources such as the Wiki or blog posts in varying degrees of correctness and up-to-date-ness appear front and center on search engines. Changing any of that involves lots of detail knowledge and attention from maintainers...

I totally agree on your perspective, but I feel like we could do a lot more integrating the content than fragmenting it. And I'd really like to hear from you or someone else on the team about what opinions they have formed regarding this fact. Any references you can give me on discussions in this regard (or also Nix documentation in general) I'd love to read them.

Again, thanks for reaching out. Feel free to ask for help if you like to get involved!

Thanks to you for your kind and extensive response! Happy New Year!

changes in the way of thinking [about documentation]

wiseh: I'm certainly open for discussing ways of thinking, and recommend you to get involved with the documentation team led by Luc Perkins to gather and provide more context. We just have to use our scarce time effectively, and balance looking at things from a distance, and working on concrete improvements. It becomes much easier to strike that balance when you have more insights into the part of the big picture that's below the surface. As said, I really value your input, because you're in a position to see the surface much better than people who have been involved for multiple years.

running into the question [...] why there are so many ways to install, what each is for, and what potential uses they might have

wiseh: Yes, this kind of information (Diátaxis calls it Explanation) absolutely needs to be covered, and has traditionally been either neglected, scattered over different manuals in various margin notes – even worse, because not subject to updates, blog posts – or more or less maintained on the wiki. That whole area still lacks a proper concept. I originally intended to focus mostly on explanations with what was supposed to be the "blessed" Nix book (now merely a stalled one of many attempts) before pivoting to what I got convinced to perceive as more pressing, practical problems. You're warmly invited to pick that up and I can make a bit of time for bouncing ideas. Again, the documentation team would be your first address to get more context or put forward your thoughts. A good way to start would be opening an issue on nix.dev where the documentation team currently maintains their agenda.

break down barriers to create documentation if it allows more people to do it and it is less dispersed. In that sense the documentation offered by search.nixos.org should be a complement to the documentation

wiseh: Fully agree, contributing to documentation should be easy and not require all too much technical knowledge. That is the current focus of the docs team efforts, and your feedback is highly valued. Please provide us with insights on where you're struggling if you want to fix things. (And thanks again for bringing it up in the first place!)

What I wanted to say is that I understand search.nixos.org as an interface, a presentation layer for what reference-style information is available. Much of that is essentially encoded in the packages themselves, so documentation more or less equals code. This is why I mentioned working on search, because that's where one can tweak how that information is presented.

I would very much like to hear the conclusions of your meetings with the search.nixos.org maintainers!

wiseh: Oh, I only submitted and discussed issues on the repo and they were addressed quickly, which was a very pleasant experience.

override options are not displayed, but that is a deeper technical issue due to how the Nix language works and how Nixpkgs is organised.

Could you explain a little more about this technical problem? I don't think there is an automatable way to provide information about this.

wiseh: The Nix language is lazily evaluated and dynamically typed, so without essentially throwing a full evaluator at Nix expressions there is no mechanical way to figure out function arguments and names in an attribute set. For Nixpkgs that means evaluating everything just to get that information. It gets even hairier if you want annotations of some sort that would serve as documentation to the language tokens, such as named function parameters.

See https://github.com/lf-/nix-doc for a third-party tool and https://github.com/NixOS/nix/pull/1652 for a discussion about adding that capability to the Nix language.

override options are not displayed, but that is a deeper technical issue due to how the Nix language works and how Nixpkgs is organised.

Could you explain a little more about this technical problem? I don't think there is an automatable way to provide information about this.

wiseh: The Nix language is lazily evaluated and dynamically typed, so without essentially throwing a full evaluator at Nix expressions there is no straightforward way to figure out function arguments and names in an attribute set. For Nixpkgs that means evaluating everything just to get that information. It gets even hairier if you want annotations of some sort that would serve as documentation to the language tokens, such as named function parameters.

But no, that information can of course be encoded and – in principle – extracted mechanically. It's just not obvious how to do it, and a robust solution is a high-value target in the ecosystem.

See https://github.com/lf-/nix-doc for a third-party tool and https://github.com/NixOS/nix/pull/1652 for a discussion about adding that capability to the Nix language.

I understood that I had to overwrite a library of his inputs for another package with extended capabilities. I would LOVE to have an official place to document that situation

wiseh: You're right, we were talking about two different things here. What you're referring to here would amount to a Tutorial (for teaching a transferrable skill) or a Guide (a recipe for applying some existing skills for some narrowly defined use case). And indeed, that should be in a highly visible place. Would be great if you found time to open a pull request on nix.dev to write down with what you learned. The Nixpkgs manual has some brief documentation on override and overrideAttrs, but it could use more and more detailed examples and a bit of explanation. Ping @NixOS/documentation-team for help getting it merged (and expect to learn many cursed details about all things Nix in the process).

I understood that I had to overwrite a library of his inputs for another package with extended capabilities. I would LOVE to have an official place to document that situation

wiseh: You're right, we were talking about two different things here. What you're referring to here would amount to a Tutorial (for teaching a transferrable skill) or a Guide (a recipe for applying some existing skills for some narrowly defined use case). And indeed, that should be in a highly visible place. Would be great if you found time to open a pull request on nix.dev to write down with what you learned. The Nixpkgs manual has some brief documentation on override and overrideAttrs, but it could use more and more detailed examples and a bit of explanation. Ping @NixOS/documentation-team for help getting your work merged (and expect to learn many cursed details about all things Nix in the process).

I'd be really happy if we could spend people of the future the time to get through that same mess.

There are differences between the terms ordering and organizing. Ordering is simply assigning a pattern. Organizing REQUIRES that pattern to facilitate access to that which is being ordered.

wiseh: Absolutely. What you describe about interacting with Nix documentation is well-known and has aptly been called tab-explosion.

I strongly agree that this has to be addressed and some sort of "information system" is needed – a set of patterns, conventions, or rules how to interlink relevant types of documentation or present it in a way that at least appears unified in a given context. There are a few considerations that make this hard to untangle, for instance keeping documentation in sync with the code, making versions explicit, and still having a low-tech way of maintaining it. I'm not convinced though that a wiki is the right tool for authoritative documentation, but it can be a good supplement.

perhaps that is where the care in making the code provide the documentation comes from; but it cannot abstract as much information from the code as the active use of the users can convey, nor does it allow as much flexibility as documentation created by the community for the community.

wiseh: This is just my opinion which I can hardly back up by evidence, so take it with a grain of salt. I think code should and can be written in a way that is easily accessible to proficient users. It's much, much more expensive to do that instead of what hackers usually do, but it's possible and almost always worthwhile in public code in heavy use. Code is just another means of communication, and one has to integrate it with natural language and other forms of communications as needed. Therefore, documentation should live as close to the working code as possible. Figuring out how to do that such that it's easy to get into and work with for users and maintainers alike is tricky though.

The problem I see with wiki-style documentation is that for code, the wiki is actually one step removed from the actual development. This is not where all the action is happening, and that's in the code.

we could do a lot more integrating the content than fragmenting it

Any references you can give me on discussions in this regard (or also Nix documentation in general) I'd love to read them.

wiseh: Here is a first version that the documentation set up on nix.dev for guiding contribtuors : https://nix.dev/contributing/documentation It's based on a few ideas that we discussed in the past year and touches on your topic of integration to the extent of encouraging people to move different types of information to the places where one would expect them. The documentation team is also striving to eventually present all official documentation in one place. Additionally, I think we should be very careful to maintain a single source of truth for every piece of information.