| 16 May 2023 |
hexa | and starts like so | 18:31:29 |
hexa | #!/nix/store/6aphgr19pzc8v416zaf6vbs7r9jsbajp-nodejs-18.16.0/bin/node
/**
* @param {string} command process to run
* @param {string[]} args command line arguments
* @returns {Promise<void>} promise
*/
const runCommand = (command, args) => {
const cp = require("child_process");
return new Promise((resolve, reject) => {
| 18:31:33 |
hexa | and I can start that interpreter | 18:31:53 |
hexa | ❯ /nix/store/6aphgr19pzc8v416zaf6vbs7r9jsbajp-nodejs-18.16.0/bin/node
Welcome to Node.js v18.16.0.
Type ".help" for more information.
| 18:32:00 |
hexa | https://github.com/django-hijack/django-hijack/blob/master/package.json#L5 | 18:36:05 |
hexa | the build command | 18:36:11 |
| 17 May 2023 |
hexa | so sorry, but I'm running into another problem I don't understand | 19:00:14 |
hexa | bokehjs> unpacking sources
bokehjs> unpacking source archive /nix/store/bibdvavvcjjm5rgs1xy2x4ckxg3jkchj-source
bokehjs> source root is source/bokehjs
bokehjs> patching sources
bokehjs> Executing npmConfigHook
bokehjs> Configuring npm
bokehjs> Validating consistency between /build/source/bokehjs/package-lock.json and /nix/store/14kzn4a79fgaknfj3nd84a3j4szxf371-bokehjs-3.1.1-npm-deps/package-lock.json
bokehjs> Making cache writable
bokehjs> Installing dependencies
bokehjs> npm WARN deprecated source-map-resolve@0.6.0: See https://github.com/lydell/source-map-resolve#deprecated
bokehjs> npm ERR! code ENOTCACHED
bokehjs> npm ERR! request to https://registry.npmjs.org/p-map failed: cache mode is 'only-if-cached' but no cached response is available.
bokehjs>
bokehjs> npm ERR! A complete log of this run can be found in:
bokehjs> npm ERR! /build/cache/_logs/2023-05-17T18_59_40_525Z-debug-0.log
bokehjs>
bokehjs> ERROR: npm failed to install dependencies
bokehjs>
bokehjs> Here are a few things you can try, depending on the error:
bokehjs> 1. Set `makeCacheWritable = true`
bokehjs> Note that this won't help if npm is complaining about not being able to write to the logs directory -- look above that for the actual error.
bokehjs> 2. Set `npmFlags = [ "--legacy-peer-deps" ]`
bokehjs>
| 19:00:18 |
hexa | npmFlags = [ "--legacy-peer-deps" ];
makeCacheWritable = true;
| 19:00:47 |
hexa | https://github.com/bokeh/bokeh/blob/3.1.1/bokehjs/package.json | 19:01:25 |
Lily Foster | In reply to @hexa:lossy.network https://github.com/bokeh/bokeh/blob/3.1.1/bokehjs/package.json npm workspaces. I haven't done a deep dive yet to figure out why npm gets too spooked to use the cache when workspaces are involved like this. The method to specify a workspace package like from https://github.com/NixOS/nixpkgs/pull/218923 may help | 19:18:32 |
Lily Foster | dotlambda: If it helps for fixing the above package, would it be okay if I separate out the change to add an npmWorkspace argument from that bitwarden-cli PR and make my own separate PR? (or alternatively if you have time to respond to the review comments, we can keep the work in that PR) | 19:21:05 |
Lily Foster | In reply to @hexa:lossy.network so sorry, but I'm running into another problem I don't understand No sorry needed, btw. Always appreciate people bringing forward issues/edge cases to help with the tooling | 19:21:52 |
hexa | In reply to @lily:lily.flowers npm workspaces. I haven't done a deep dive yet to figure out why npm gets too spooked to use the cache when workspaces are involved like this. The method to specify a workspace package like from https://github.com/NixOS/nixpkgs/pull/218923 may help so another modification to build-npm-package? 😄 | 19:22:23 |
Lily Foster | In reply to @hexa:lossy.network so another modification to build-npm-package? 😄 Yeah, but hopefully a minor one. I'll try to split out the workspace flag hack and see if it would help in this case. Would you mind sharing the whole derivation you're working with? | 19:24:18 |
hexa | in a bit 😄 | 19:24:37 |
hexa | it's an unlanded PR rebased and locally updated to a newer version during rebase 🤡 | 19:24:53 |
Lily Foster | (alternatively I can just go figure out why npm gets too spooked to use the cache with workspaces like this -- their lack of instrumentation/debug messaging really makes it difficult to track down where decisions like "oh the cache is bad, time to refetch" are made) | 19:25:14 |
hexa | there is the django-hijack issue above, if you want to knock yourself out 😄 | 19:25:30 |
Lily Foster | (especially since it doesn't log that it even decides the cache is bad) | 19:25:39 |
Lily Foster | In reply to @hexa:lossy.network there is the django-hijack issue above, if you want to knock yourself out 😄 Oh yeah I need to come back to that one. I'll do that right now | 19:25:48 |
Lily Foster | In reply to @hexa:lossy.network
node_modules/webpack/bin/webpack.js: a /nix/store/6aphgr19pzc8v416zaf6vbs7r9jsbajp-nodejs-18.16.0/bin/node script, ASCII text executable
So this is right. But the setup.py also runs npm ci right before the build script which undoes the fixing that the hook already did (ci is for clean-installing deps, but the deps should already be installed) | 19:56:58 |
Lily Foster | This diff to your derivation makes it work:
diff --git a/pkgs/development/python-modules/django-hijack/default.nix b/pkgs/development/python-modules/django-hijack/default.nix
index ff0eb44ef44..34457b923c6 100644
--- a/pkgs/development/python-modules/django-hijack/default.nix
+++ b/pkgs/development/python-modules/django-hijack/default.nix
@@ -32,6 +32,7 @@ buildPythonPackage rec {
postPatch = ''
substituteInPlace setup.py \
+ --replace 'cmd = ["npm", "ci"]' 'cmd = ["true"]' \
--replace 'f"{self.build_lib}/{name}.mo"' 'f"{name}.mo"'
'';
| 19:57:08 |
hexa | ok | 19:58:35 |
hexa | cool | 19:58:37 |
Lily Foster | (if you --keep-failed and check the webpack.js file after the build fails, you'll notice the shebang goes back to #!/usr/bin/env node) | 19:59:26 |
Lily Foster | (despite being correct at preBuild) | 19:59:37 |
hexa | thanks! | 20:07:34 |
Lily Foster | For once it wasn't npm's fault, surprisingly | 20:07:55 |
Lily Foster | In reply to @hexa:lossy.network it's an unlanded PR rebased and locally updated to a newer version during rebase 🤡 If you want to test it yourself, add a npmWorkspace argument with the npm workspace package you are wanting to build and apply this diff locally:
diff --git a/pkgs/build-support/node/build-npm-package/default.nix b/pkgs/build-support/node/build-npm-package/default.nix
index 1c3fb6a74ef..0aa5ade9dc3 100644
--- a/pkgs/build-support/node/build-npm-package/default.nix
+++ b/pkgs/build-support/node/build-npm-package/default.nix
@@ -30,6 +30,8 @@
, npmBuildFlags ? [ ]
# Flags to pass to `npm pack`.
, npmPackFlags ? [ ]
+ # Value for npm `--workspace` flag and directory in which the files to be installed are found.
+, npmWorkspace ? null
, ...
} @ args:
@@ -45,6 +47,8 @@ in
stdenv.mkDerivation (args // {
inherit npmDeps npmBuildScript;
+ npmFlags = args.npmFlags ++ lib.optionals (npmWorkspace != null) [ "--workspace" npmWorkspace ];
+
nativeBuildInputs = nativeBuildInputs ++ [ nodejs npmConfigHook npmBuildHook npmInstallHook ];
buildInputs = buildInputs ++ [ nodejs ];
diff --git a/pkgs/build-support/node/build-npm-package/hooks/npm-install-hook.sh b/pkgs/build-support/node/build-npm-package/hooks/npm-install-hook.sh
index 59ea2da993d..afc911cd021 100644
--- a/pkgs/build-support/node/build-npm-package/hooks/npm-install-hook.sh
+++ b/pkgs/build-support/node/build-npm-package/hooks/npm-install-hook.sh
@@ -13,7 +13,7 @@ npmInstallHook() {
while IFS= read -r file; do
local dest="$packageOut/$(dirname "$file")"
mkdir -p "$dest"
- cp "$file" "$dest"
+ cp "${npmWorkspace:-.}/$file" "$dest"
done < <(@jq@ --raw-output '.[0].files | map(.path) | join("\n")' <<< "$(npm pack --json --dry-run $npmPackFlags "${npmPackFlagsArray[@]}" $npmFlags "${npmFlagsArray[@]}")")
while IFS=" " read -ra bin; do
@@ -22,7 +22,7 @@ npmInstallHook() {
done < <(@jq@ --raw-output '(.bin | type) as $typ | if $typ == "string" then
.name + " " + .bin
elif $typ == "object" then .bin | to_entries | map(.key + " " + .value) | join("\n")
- else "invalid type " + $typ | halt_error end' package.json)
+ else "invalid type " + $typ | halt_error end' "${npmWorkspace:-.}/package.json")
local -r nodeModulesPath="$packageOut/node_modules"
| 20:17:45 |