Does buildNpm have troubles with sourcehut git sources?
I'm trying to build https://gitdab.com/cadence/out-of-your-element but I'm getting:

out-of-your-element> unpacking sources
out-of-your-element> unpacking source archive /nix/store/8xka9inmpaq7qb0g1s80r9qvnilidpb8-source
out-of-your-element> source root is source
out-of-your-element> patching sources
out-of-your-element> Executing npmConfigHook
out-of-your-element> Configuring npm
out-of-your-element> Validating consistency between /build/source/package-lock.json and /nix/store/ydflkz6y8r7a1wm0q29r6nqmgv3n2wyj-out-of-your-element-1.2-npm-deps/package-lock.json
out-of-your-element> Fixing lockfile
out-of-your-element> Making cache writable
out-of-your-element> Installing dependencies
out-of-your-element> npm verb cli /nix/store/x8vrc1c6r8nn454whr99bxc5hdy5ybjf-nodejs-18.18.0/bin/node /nix/store/x8vrc1c6r8nn454whr99bxc5hdy5ybjf-nodejs-18.18.0/bin/npm
out-of-your-element> npm info using npm@9.8.1
out-of-your-element> npm info using node@v18.18.0
out-of-your-element> npm verb title npm ci
out-of-your-element> npm verb argv "ci" "--ignore-scripts" "--loglevel" "verbose"
out-of-your-element> npm verb logfile logs-max:10 dir:/build/cache/_logs/2023-10-14T03_08_27_400Z-
out-of-your-element> npm verb logfile /build/cache/_logs/2023-10-14T03_08_27_400Z-debug-0.log
out-of-your-element> npm WARN tarball tarball data for discord-markdown@git+https://git.sr.ht/~cadence/nodejs-discord-markdown#abc56d544072a1dc5624adfea455b0e902adf7b3 (null) seems to be corrupted. Trying again.
out-of-your-element> npm WARN tarball tarball data for discord-markdown@git+https://git.sr.ht/~cadence/nodejs-discord-markdown#abc56d544072a1dc5624adfea455b0e902adf7b3 (null) seems to be corrupted. Trying again.
out-of-your-element> npm http fetch GET 200 https://codeload.github.com/cloudrac3r/mixin-deep/tar.gz/2dd70d6b8644263f7ed2c1620506c9eb3f11d32a 263ms (cache stale)
out-of-your-element> npm http fetch GET 200 https://codeload.github.com/cloudrac3r/tap-out/tar.gz/1b4ec6084aedb9f44ccaa0c7185ff9bfd83da771 242ms (cache stale)
out-of-your-element> npm http fetch GET 200 https://codeload.github.com/cloudrac3r/tap-dot/tar.gz/9dd7750ececeae3a96afba91905be812b6b2cc2d 244ms (cache stale)
out-of-your-element> npm http fetch GET 200 https://codeload.github.com/cloudrac3r/giframe/tar.gz/1630f4d3b2bf5acd197409c85edd11e0da72d0a1 274ms (cache stale)
out-of-your-element> npm verb stack Error: An unknown git error occurred
out-of-your-element> npm verb stack     at makeError (/nix/store/x8vrc1c6r8nn454whr99bxc5hdy5ybjf-nodejs-18.18.0/lib/node_modules/npm/node_modules/@npmcli/git/lib/make-error.js:28:13)
out-of-your-element> npm verb stack     at /nix/store/x8vrc1c6r8nn454whr99bxc5hdy5ybjf-nodejs-18.18.0/lib/node_modules/npm/node_modules/@npmcli/git/lib/spawn.js:37:26
out-of-your-element> npm verb stack     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
out-of-your-element> npm verb cwd /build/source
out-of-your-element> npm verb Linux 6.5.5
out-of-your-element> npm verb node v18.18.0
out-of-your-element> npm verb npm  v9.8.1
out-of-your-element> npm ERR! code ENOENT
out-of-your-element> npm ERR! syscall spawn git
out-of-your-element> npm ERR! path git
out-of-your-element> npm ERR! errno -2
out-of-your-element> npm ERR! enoent An unknown git error occurred
out-of-your-element> npm ERR! enoent This is related to npm not being able to find a file.
out-of-your-element> npm ERR! enoent
out-of-your-element> npm verb exit -2
out-of-your-element> npm verb unfinished npm timer reify 1697252907725
out-of-your-element> npm verb unfinished npm timer reify:unpack 1697252907754
out-of-your-element> npm verb unfinished npm timer reifyNode:node_modules/discord-markdown 1697252907764
out-of-your-element> npm verb code -2

In reply to @dandellion:dodsorf.as
If I give it git through nativeBuildInputs it'll fail to reach out of its sandbox while running git --no-replace-objects ls-remote https://git.sr.ht/~cadence/nodejs-discord-markdown which seems super weird

In reply to @ahbk:matrix.org
Is there a reason mkYarnPackage doesn't cache or am I doing something wrong? All dependecies are cached but the package itself isn't so it rebuilds every time

In reply to @lily:lily.flowers
Cache like upload to cachix or something?

In reply to @lily:lily.flowers
Cache like upload to cachix or something?

No sorry not cached. I'm not sure I understand this, but I when the flake is changed, I guess the yarn package has to be recalculated or evaluated which takes some time (about 15 seconds). This is different from the other packages in the flake which are evaluated almost immediately if they haven't been changed. I experienced this as a cache miss but I realize now that it is not about that. Here's the flake:

https://github.com/ahbk/ahbk/blob/main/flake.nix

No sorry not cached. I'm not sure I understand this, but when the flake is changed, I guess the yarn package has to be recalculated or evaluated which takes some time (about 15 seconds). This is different from the other packages in the flake which are evaluated almost immediately if they haven't been changed. I experienced this as a cache miss but I realize now that it is not about that. Here's the flake:

https://github.com/ahbk/ahbk/blob/main/flake.nix

No sorry not cached. I'm not sure I understand this, but when the flake is changed I think the yarn package has to be recalculated or evaluated which takes some time (about 15 seconds), even if the change didn't affect the package. This is different from the other packages in the flake which are evaluated almost immediately if they haven't been changed. I experienced this as a cache miss but I realize now that it is not about that. Here's the flake:

https://github.com/ahbk/ahbk/blob/main/flake.nix

No sorry not cached. I'm not sure I understand this, but when the flake is changed I think the yarn package has to be recalculated or evaluated which takes some time (about 15 seconds), even if the change didn't affect the package. This is different from the other packages in the flake which are evaluated almost immediately if they haven't been changed. I experienced this as a cache miss but I realize now that it is not about that. Here's the flake:

https://github.com/ahbk/ahbk/blob/main/flake.nix#L39

In reply to @ahbk:matrix.org

No sorry not cached. I'm not sure I understand this, but when the flake is changed I think the yarn package has to be recalculated or evaluated which takes some time (about 15 seconds), even if the change didn't affect the package. This is different from the other packages in the flake which are evaluated almost immediately if they haven't been changed. I experienced this as a cache miss but I realize now that it is not about that. Here's the flake:

https://github.com/ahbk/ahbk/blob/main/flake.nix#L39

In reply to @lily:lily.flowers
Hmmmm if i remember i'll try to investigate later of it's just our fetcher not doing something right or if npm is the one who is choking on it

I made an impure derivation, and now it's failing with:

out-of-your-element> npm ERR! command git --no-replace-objects ls-remote https://git.sr.ht/~cadence/nodejs-discord-markdown
out-of-your-element> npm ERR! fatal: unable to access 'https://git.sr.ht/~cadence/nodejs-discord-markdown/': OpenSSL/3.0.10: error:16000069:STORE routines::unregistered scheme
out-of-your-element> npm verb exit 128
out-of-your-element> npm verb unfinished npm timer reify 1697324205527
out-of-your-element> npm verb unfinished npm timer reify:unpack 1697324205560
out-of-your-element> npm verb unfinished npm timer reifyNode:node_modules/discord-markdown 1697324205573
out-of-your-element> npm verb code 128

I don't know if it's very relevant, but it's kind of sus

In reply to @dandellion:dodsorf.as

I made an impure derivation, and now it's failing with:

out-of-your-element> npm ERR! command git --no-replace-objects ls-remote https://git.sr.ht/~cadence/nodejs-discord-markdown
out-of-your-element> npm ERR! fatal: unable to access 'https://git.sr.ht/~cadence/nodejs-discord-markdown/': OpenSSL/3.0.10: error:16000069:STORE routines::unregistered scheme
out-of-your-element> npm verb exit 128
out-of-your-element> npm verb unfinished npm timer reify 1697324205527
out-of-your-element> npm verb unfinished npm timer reify:unpack 1697324205560
out-of-your-element> npm verb unfinished npm timer reifyNode:node_modules/discord-markdown 1697324205573
out-of-your-element> npm verb code 128

I don't know if it's very relevant, but it's kind of sus

In reply to @dandellion:dodsorf.as
fwiw running nix develop on the derivation, and then manually running the npmConfigHook and such does succesfully build it

In reply to @lily:lily.flowers
That's expected since dev shell lets npm call out to the internet when it's not supposed to need to (npm gets very very happy about silently deciding something is wrong and deleting and redownloading from the internet....)

In reply to @felschr:matrix.org
Lily Foster, I guess buildNpmPackage could add a npmWorkspaces option to simplify building multiple workspaces. It's not really an issue, though, to just build the other packages in a preBuild.