In reply to @j-k:matrix.org

Slightly related: I just noticed /nix/var/log/nix/drvs/ has a 2 letter prefix sub dir pattern

λ ls /nix/var/log/nix/drvs/
 00   0s   1l   2f   38   42   4w   5n   6h   7a # ... etc ...

/nix/store is stored locally, rarely listed and mostly accessed by the entry name. That's why we get get away with FS choosing optimal representation for it (hash tree usually).
Nixpkgs is stored in Git and often listed in editors and UIs, all of them are not great with huge directories, so we should optimize for smaller ones.

I wonder why such nested structure was chosen for logs though.

In reply to @kevincox:matrix.org

I'm thinking something like this. (For a flat solution)

all-packages = builtin.proxyAttrs {
  lookup = name: pkgs.callPackage (import "pkgs/${name}.nix");
  list-keys = # List all packages and return a list or something here.
};

Then for the common case of nix-build -A foo it just calls lookup "foo" and only that package is loaded.

I really like that idea. Especially as it allows to add a package by just dropping a file somewhere. Having to edit some all-packages.nix is really unnecessary overhead.

Assuming overrides are still a thing at the time when this gets implemented, we'd have to find a new solution for the default callPackage overrides that are currently done in all-packages.nix for some packages.
I see the following options for this:

1. Create one central overrides file for default callPackage overrides (I think this is a bad idea)
2. Manage the overrides within the .nix of each package, though then it requires a reference to pkgs.
3. Deprecate in-line overrides all-together and enforce the creation of an actual individual package for each required package variation. This would make every existing package variation referenceable by an actual attribute name, which is great for further overriding by the user.

For example, if we have packages hello and foobar, where hello depends on an overridden foobar, we could have the following files:

/fo/ob/foobar.nix
/he/ll/hello.nix
/he/ll/foobar.nix  -  (the overridden foobar package for hello)

/he/ll/foobar.nix could look like this:

{foobar,}:
foobar.overrideAttrs (old: {
   pname = "foobar-for-hello";
})

In reply to @k900:0upti.me
Maybe in a special top-level directory

I noticed that I sparked quite a discussion 😄 So I ideally would see:

• Folder structure representing attrs structure. If package is in top level - under root directory. If there is a package group (gnome, pythonPackages, etc) - then it should be under directory with the same name
• I got a feeling that all-packages.nix is necessary otherwise we would be impacting performance but I have not tested it (also we could autogenerate such file)
• all these overrides should be separated from all-packages.nix I'd say one file for 'dumb package definitions' and another for overwrites
• Consider exposing blueprints https://github.com/gytis-ivaskevicius/nix-patterns/blob/master/blueprints/flake.nix#L7-L10 instead of derivations and mapAttrs when necessary (is it more work than its worth?)0

nix-repl> length (attrNames pkgs)
16563

I wonder if we need overrides in nixpkgs. Maybe we can give every modified package a name and let the called package use that name. (basically no more python = python38). But that does add some API instability for user overrides. Maybe a multi-level approach can work?

I like the look of blueprints but how does this work for user-overrides?