| 31 Oct 2023 |
 Lily Foster | (i mostly know what needs to be done, but it's a matter of doing it and the initial energy just to convince myself to get into node work can be a lot sometimes) | 19:51:00 |
 Wanja Hentze | we also have patches for that to make it retry with backoff that we intend to mainline ahem | 19:51:16 |
| Lily Foster | In reply to @whentze:matrix.org
that works well but the need to maintain the npmDepsHash is quite unfortunate. for larger projects the prefetcher takes forever and also eventually runs into npm rate limits We're fixing the npmDepsHash deal for out-of-tree stuff by having something like importCargoLock for npm lockfiles. And can you post a build log about the rate limits? I've never seen rate limits | 19:51:32 |
| Lily Foster | In reply to @whentze:matrix.org
we also have patches for that to make it retry with backoff that we intend to mainline ahem does it not already retry with backoff? | 19:51:45 |
| Wanja Hentze | In reply to @lily:lily.flowers
I mean I work on nixpkg's buildNpmPackage, and we basically work around npm bugs. The other decent one would be floco since they just reimplement npm's functionality, but apparently they're using a nix plugin now which just feels cursed? I've not seen enough about the dream2nix one, but i've heard it works more reliably than any of the others. The rest are outdated and highly questionable the nix plugin is optional fwiw. you don't need it for floco itself | 19:51:45 |
 Marie | i've never had buildNpmPackage run into rate limits, only the curl issue we had | 19:51:59 |
| Wanja Hentze | nope | 19:52:02 |
| Lily Foster | In reply to @whentze:matrix.org
nope how are you testing that? | 19:52:11 |
| Lily Foster | i wrote the code for that. and tested it | 19:52:15 |
| Lily Foster | and it got merged ages ago | 19:52:20 |
| Wanja Hentze | huh | 19:52:20 |
| Lily Foster | and it does fetch parallel and always has | 19:52:37 |
| Wanja Hentze | we just put it in a mid-sized react project | 19:52:42 |
| Lily Foster | (i actually recently made it respect NIX_BUILD_CORES or whatever the env var is not long ago) | 19:52:54 |
| Wanja Hentze | yes it does fetch in parallel, that is part of the problem | 19:52:55 |
| Lily Foster | What nixpkgs branch are you working off of? | 19:53:08 |
| Wanja Hentze | uhh unstable but this was a couple weeks ago | 19:53:29 |
| Wanja Hentze | I could try again | 19:53:36 |
| Lily Foster | rev? | 19:53:58 |
| Wanja Hentze | dunno, I'm on the phone | 19:54:07 |
| Lily Foster | these fixes were all merged more than a few weeks ago i thought | 19:54:08 |
| Wanja Hentze | sry | 19:54:13 |
| Lily Foster | no it's okay! | 19:54:16 |
| Lily Foster | i'm just confused 😅 | 19:54:20 |
| Wanja Hentze | very possible that they have | 19:54:25 |
| Lily Foster | Well I think the retries failing for response bodies issue was more recent | 19:54:46 |
| Wanja Hentze | another weird failure mode we saw: the fetcher seemed to ignore HTTP 4XX from the remote and would just try to decode the body (which was text/html saying "permission denied" or sth) as a tarball | 19:55:22 |
| Wanja Hentze | maybe that's fixed too | 19:55:36 |
| Lily Foster | quite possibly? i do appreciate you bringing these issues up though | 19:55:50 |
| Lily Foster | do you think when you run into them you could open nixpkgs issues and ping me and/or the node team on thme? | 19:56:01 |
