| 10 Aug 2021 |
 niksnut | In reply to @Las:matrix.org
I tried this on my own machine, and it did indeed take ages (so long I interrupted it), but time git clone git://github.com/NixOS/nixpkgs --depth 1 took 10.31 seconds according to time. Yes, that's what ?shallow= does. | 13:32:06 |
| niksnut | hm | 13:32:18 |
 Las | I wonder why it takes so long then | 13:32:20 |
 balsoft | BTW I can confirm that Nix' fetcher is way slower than fetching manually | 13:32:38 |
| niksnut | Nix just calls git, it doesn't fetch itself | 13:33:29 |
| niksnut | Looks like we're not passing --depth 1 | 13:34:39 |
| Las | isn't it not shallow then? | 13:34:48 |
| niksnut | yeah | 13:36:14 |
 pamplemousse | In reply to @sternenseemann:systemli.org
pamplemousse: also these usually use binutils' ld.bfd which sometimes causes weird problems linking C++, so pkgsLLVM.stdenv may be worth a try which uses lld I think https://github.com/NixOS/nix/pull/4996/files#diff-206b9ce276ab5971a2489d75eb1b12999d4bf3843b7988cbe8d687cfde61dea0L600-R631 makes nix develop automatically pick the stdenv corresponding to what I need (I do nix develop .#nix-clang11Stdenv. | 16:10:34 |
| pamplemousse | sterni (he/him): and I get an /nix/store/qvc6cz5d43jhhm4a3r48xkrc82xh1s95-binutils-2.35.1/bin/ld: cannot find -lc++abi when trying to use llvmPackages_11.libcxx | 17:50:04 |
 sterni | libcxx or libcxxStdenv? | 17:50:41 |
| pamplemousse | libcxx
got an /nix/store/qvc6cz5d43jhhm4a3r48xkrc82xh1s95-binutils-2.35.1/bin/ld: cannot find -lc++ with libcxxStdenv
(both from llvmPackages_11) | 17:59:58 |
| sterni | what were you doing exactly? libcxxabi is in extraPackages of libcxxClang, so it should not fail in this way | 18:01:23 |
 Vladimír Čunát | In reply to @Las:matrix.org
The issue is you can't check commit signatures when using the github fetcher Does it make sense to check git signature unless you have the whole history? (up to that commit; in particular, no --depth stuff) | 18:49:20 |
| Las | In reply to @vcunat:matrix.org
Does it make sense to check git signature unless you have the whole history? (up to that commit; in particular, no --depth stuff) I'm not sure. I found other ways of heavily reducing the time it takes to fetch Nixpkgs, specifically --filter=tree:0, but it still takes 4 times as much time. | 18:56:55 |
| Las | With a shallow repository I can still see that the commit is signed | 18:57:16 |
| Las | But I'm not sure if it can be trusted | 18:57:22 |
| 11 Aug 2021 |
 Kha | In reply to @Las:matrix.org
The issue is you can't check commit signatures when using the github fetcher And also the github fetcher is rate limited (60 requests/h per public IP(!)) while the git fetcher isn't, right? | 09:08:16 |
| Kha | Basically github: works great until it doesn't anymore | 09:08:29 |
| Kha | In reply to @Las:matrix.org
The issue is you can't check commit signatures when using the github fetcher * And also the github fetcher is rate limited (60 requests/h per public IP(!) without GH token) while the git fetcher isn't, right? | 09:09:05 |
| Las | That is very low. With shallow cloning I honestly don't see the need for it anymore. | 11:09:40 |
| sterni | who runs a garbage collect 60 times an hour is the question? | 13:05:34 |
| sterni | I don't think there's any conceivable scenario where this is an issue | 13:05:45 |
 andi- | It isn't about 60 times an hour. Divide that by the number of sources you might require for your build (and perhaps you are changing some). I usually run into that limit when I update my source pinnings of my infra repo twice in the same hour. | 13:09:27 |
| Kha | Then divide that by the number of machines/people behind a single IP, e.g. at an office or university | 13:13:57 |
| andi- | Yeah and all of that because GH doesn't deploy IPv6 :) | 13:14:48 |
 nixinator | rate limiting will get worse. | 13:17:32 |
| nixinator | not better. | 13:17:35 |
| nixinator | In reply to @andi:kack.it
Yeah and all of that because GH doesn't deploy IPv6 :) what has v6 got do with rate limiting? | 13:18:14 |
| andi- | They could either rate limit on per /64 or per unique address on the side of GitHub. Worst case they use a /56 or even a /48 as prefix for common rate limit but best case they'd limit it on a L2 segment and thus allowing multiple customers on e.g. CGNAT do not interfere with each others ratelimits.
In german cable networks CGNAT is very dominant and often you run into ratelimits with GitHub as the outbound IPv4 NAT address might be used by many users within the same area. | 13:20:02 |
