| 25 Dec 2025 |
Lotte (it/its)/Cinny (she/her) | git handles big repos more poorly than they need to be
although nixpkgs still needs a significant amount of the repo checked out to function i think
| 15:35:06 |
KFears (they/them) | Nixpkgs is given as an example of a big tepo that is also kind of a package registry | 15:35:14 |
KFears (they/them) | Not as an example of the fully traditional package registry | 15:35:25 |
KFears (they/them) | Point here being is that if you push enough shit into git, it will fall apart | 15:35:44 |
Lotte (it/its)/Cinny (she/her) | if IFD was cheap i think nixpkgs could be way lighter than a git repo checkout or a tarball | 15:39:45 |
K900 | Git is fine at nixpkgs scale honestly | 16:22:25 |
K900 | Github on the other hand | 16:22:32 |
piegames | I think that post is less relevant for Nixpkgs itself and more for whatever comes after Flakes | 17:08:33 |
emily | I think this is probably LLM slop but as someone who was on the call with GitHub I would not say this is at all a reasonable takeaway | 17:15:32 |
emily | (GitHub scaling issues ≠ Git scaling issues, the acute GitHub scaling issues were declared provisionally resolved in the same update that talked about them and confirmed as definitely resolved in the next update, the size of the Nixpkgs repo itself is trivial factor compared to the development activity, and none of the factors in "The pattern" are remotely relevant) | 17:18:46 |
emily | (actually the call started out with the GitHub engineer being like "is this maybe using Git as a database?" and within 5 minutes of explanation we were all on the same page of "no, it's genuinely a codebase and not a database") | 17:20:32 |
emily | I'd say the actual painful-ish thing is just having to fetch all of Nixpkgs to evaluate a single package, but that's very difficult to fix (if IFD was really good it might work yeah) | 17:21:16 |
emily | (FWIW, the Linux and Chromium repositories are still bigger than Nixpkgs. they just don't use GitHub forks for development so wouldn't run into GitHub backend scaling issues) | 17:22:02 |
KFears (they/them) | In reply to @emilazy:matrix.org I'd say the actual painful-ish thing is just having to fetch all of Nixpkgs to evaluate a single package, but that's very difficult to fix (if IFD was really good it might work yeah) I don't know what it is, but fetching Nixpkgs tarball from Github is monstrously slow | 17:52:25 |
Arian | In reply to @kfears:matrix.org I don't know what it is, but fetching Nixpkgs tarball from Github is monstrously slow Are you on macos | 17:52:40 |
KFears (they/them) | It takes 20s+ on my setup, despite the tarball being 50-60MB | 17:52:56 |
Arian | It's a bug in Lix /lib archive we've been hunting down. It takes 5 minutes to fetch for me | 17:53:08 |
Unhooked Spirits | I'm on macOS and that sounds about right to me, too. Maybe 3 minutes for me. | 17:53:43 |
Arian | Almost all slowness with nixpkgs is in the unpack phase. Even not on macos | 17:53:44 |
Arian | Might be a bit better on Nix due to the new VFS stuff | 17:54:05 |
KFears (they/them) | It's absurd that it takes so long, and it makes me think there's something about Github specifically being very slow with the tarball downloads that causes very visible UX issues | 17:54:32 |
Arian | Has nothing to do with GitHub. The download part is fast. The unpack phase is slow | 17:54:56 |
KFears (they/them) | In reply to @arianvp:matrix.org Are you on macos Not, I'm not talking about recent Lix bug on MacOS, I'm talking about general Linux experience | 17:55:04 |
Arian | The tarball API of GitHub has been flakey though since like 10 nov | 17:55:36 |
Arian | I'm seeing a lot of 504s recently | 17:55:42 |
toonn | I don't remember who it was but there was some work on fetching partial tarballs using some feature of zstd, no? | 17:58:56 |
rosssmyth | You can try https://nixos.org/channels/nixos-unstable/nixexprs.tar.xz | 18:02:39 |
Ramses 🇵🇸 | In reply to @rosssmyth:matrix.org You can try https://nixos.org/channels/nixos-unstable/nixexprs.tar.xz This makes the download a lot faster, I switched to it a couple of months ago. Unpacking is still really slow though indeed | 19:39:50 |
llakala | In reply to @emilazy:matrix.org I think this is probably LLM slop but as someone who was on the call with GitHub I would not say this is at all a reasonable takeaway ty very much for your nixpkgs core team work btw. you guys are the thankless heroes helping shit actually get better | 19:50:07 |
leona | (please use channels.nixos.org, not the redirect) | 19:53:13 |