| 10 Jul 2022 |
 K900 | And then it's ab/cd/abcd.json | 15:50:43 |
 infinisil | Btw if we're concerned about performance, there's also the option of adding Nix builtins to speed up certain operations | 15:51:20 |
 kevincox | Assuming we still have .nix that seems unnecessary. It would be interesting to see their rationale. | 15:51:31 |
| infinisil | Maybe even a builtins.prefixedAutocall | 15:51:32 |
| kevincox | I think some sort of ProxyAttrs that allowed overriding lookup and iteration would be the generic option. But we could also just hardcode the lookup scheme that we pick. | 15:52:26 |
| infinisil | Oh I see, so you mean to allow people to specify some file outside of nixpkgs for a specific attribute for development? | 15:53:27 |
| infinisil | I think that's very unproblematic, an overlay can already be used for that | 15:54:31 |
| kevincox | I'm thinking something like this. (For a flat solution)
all-packages = builtin.proxyAttrs {
lookup = name: pkgs.callPackage (import "pkgs/${name}.nix");
list-keys = # List all packages and return a list or something here.
};
Then for the common case of nix-build -A foo it just calls lookup "foo" and only that package is loaded.
| 15:56:13 |
| infinisil | Ah! Yes that's an interesting idea and could have many other useful applications | 15:57:24 |
| kevincox | To all other Nix code it would behave like a regular attrs of all packages except as long as you never do something that requires listing attributes it doesn't have to build the full set. | 15:57:29 |
| kevincox | (IDK if we need other primitive operations but they can be added easily. We can also do caching to ensure that it is pure) | 15:58:11 |
| kevincox | Yeah, for nixpkgs we can make as complicated of a lookup as we like, include 1 or 2 level and whatnot. We can also fall back to the current all-packages.nix attr for transition. | 15:59:05 |
| kevincox | This could also be polyfilled for older NIxes by eagerly computing the real attrs. | 16:07:43 |
| kevincox | * This could also be polyfilled for older NIxes by eagerly computing the a attrset. | 16:09:26 |
| kevincox | * This could also be polyfilled for older NIxes by eagerly computing the resulting attrset. | 16:17:37 |
| 11 Jul 2022 |
 j-k | Slightly related: I just noticed /nix/var/log/nix/drvs/ has a 2 letter prefix sub dir pattern
λ ls /nix/var/log/nix/drvs/
00 0s 1l 2f 38 42 4w 5n 6h 7a # ... etc ...
| 11:01:01 |
|  Wanja Hentze joined the room. | 12:39:14 |
|  hexa joined the room. | 12:45:26 |
|  profpatsch joined the room. | 12:45:49 |
|  Andreas Schrägle joined the room. | 12:49:24 |
|  toonn joined the room. | 13:31:16 |
|  Winter (she/her) joined the room. | 15:18:42 |
|  kayla.fire joined the room. | 18:48:48 |
 Sandro 🐧 | In reply to @j-k:matrix.org
Slightly related: I just noticed /nix/var/log/nix/drvs/ has a 2 letter prefix sub dir pattern
λ ls /nix/var/log/nix/drvs/
00 0s 1l 2f 38 42 4w 5n 6h 7a # ... etc ...
Really common for hash based directory structures to keep the amount of things in a directory smaller | 23:54:06 |
| Sandro 🐧 | Because if you have a lot a lot of files in a directory listing that gets show | 23:54:32 |
| Sandro 🐧 | * Because if you have a lot a lot of files in a directory listing that gets slow | 23:54:37 |
| 12 Jul 2022 |
| kevincox | /nix/store laughs. | 00:42:06 |
| kevincox | But honestly it is generally better to let the storage system do something optimal. Especially for this case where the prefix will depend on human-chosen names.
However we know at least that GitHub handles this poorly and that Git doesn't do anything clever at the moment. So we do need to meet our tools. | 00:43:23 |
 yorik.sar | /nix/store is stored locally, rarely listed and mostly accessed by the entry name. That's why we get get away with FS choosing optimal representation for it (hash tree usually).
Nixpkgs is stored in Git and often listed in editors and UIs, all of them are not great with huge directories, so we should optimize for smaller ones.
I wonder why such nested structure was chosen for logs though. | 05:40:25 |
 DavHau | In reply to @kevincox:matrix.org
I'm thinking something like this. (For a flat solution)
all-packages = builtin.proxyAttrs {
lookup = name: pkgs.callPackage (import "pkgs/${name}.nix");
list-keys = # List all packages and return a list or something here.
};
Then for the common case of nix-build -A foo it just calls lookup "foo" and only that package is loaded.
I really like that idea. Especially as it allows to add a package by just dropping a file somewhere. Having to edit some all-packages.nix is really unnecessary overhead.
Assuming overrides are still a thing at the time when this gets implemented, we'd have to find a new solution for the default callPackage overrides that are currently done in all-packages.nix for some packages.
I see the following options for this:
- Create one central overrides file for default callPackage overrides (I think this is a bad idea)
- Manage the overrides within the
.nix of each package, though then it requires a reference to pkgs.
- Deprecate in-line overrides all-together and enforce the creation of an actual individual package for each required package variation. This would make every existing package variation referenceable by an actual attribute name, which is great for further overriding by the user.
For example, if we have packages hello and foobar, where hello depends on an overridden foobar, we could have the following files:
/fo/ob/foobar.nix
/he/ll/hello.nix
/he/ll/foobar.nix - (the overridden foobar package for hello)
/he/ll/foobar.nix could look like this:
{foobar,}:
foobar.overrideAttrs (old: {
pname = "foobar-for-hello";
})
| 08:47:07 |
