!NhAsaYbbgmzHtXTPQJ:funklause.de

Nix NodeJS

205 Members
59 Servers

Load older messages

SenderMessageTime
17 Nov 2023
@adis:blad.is@adis:blad.is

I had this idea today for a radically simple ingestion method for package-lock.json.
The idea is:

  • Map over package-lock.json packages, invoke correct fetcher
  • Update package.json & package-lock.lock to point to store paths
  • npm install

I have a PoC here: https://github.com/adisbladis/fetch-package-lock/blob/master/flake.nix#L78-L82

I'm wondering if there is anything obvious I've missed? It seems to easy/simple.

08:45:12
@adis:blad.is@adis:blad.isIt certainly seems to be working alright, even for some larger stuff09:14:37
@c0ba1t:matrix.orgCobaltHmm, you'd need to alter the source, right? Or do you plan to replace the package lock with a linked version?09:53:57
@adis:blad.is@adis:blad.is
In reply to @c0ba1t:matrix.org
Hmm, you'd need to alter the source, right? Or do you plan to replace the package lock with a linked version?
I replace it with one where all the URLs are substituted with Nix store paths 		09:55:03
@c0ba1t:matrix.orgCobaltOkay, that sounds nice. That might also be a neat way to work around the occasional npm cache re-download issues.09:58:34
@whentze:matrix.orgWanja Hentze adisbladis: we have a weird cursed JS codebase at work I could try this on 10:27:22
@whentze:matrix.orgWanja Hentze it has several package.json files in the repo but is not a workspace 10:27:35
@adis:blad.is@adis:blad.is
In reply to @whentze:matrix.org
adisbladis: we have a weird cursed JS codebase at work I could try this on
Cool :) 		11:31:16
@ian.bridgy:matrix.orgian.bridgy joined the room.14:43:37
@minallwc:matrix.orgMiguel Suarez joined the room.18:22:25
@minallwc:matrix.orgMiguel SuarezHello Nix Community!18:22:56
@minallwc:matrix.orgMiguel SuarezIs there a way to have a development environment created from a single file?, for example, entering an angular project and running a nix related command would give me a specific docker postgresql database and specific nodejs and npm versions that would go with the Angular project18:23:42
@c0ba1t:matrix.orgCobaltYou might want to look at arion18:32:26
@marie:marie.cologneMariei think https://devenv.sh/ is probably the better choice18:32:49
@c0ba1t:matrix.orgCobalt* You might want to look at arion, https://docs.hercules-ci.com/arion/18:33:13
@keiichi:matrix.org@keiichi:matrix.orghttps://numtide.github.io/devshell/ too18:40:55
@szucsitg:matrix.orgszucsitgI love devenv. It helps you get started, making the learning curve to nix much more bearable19:33:01
18 Nov 2023
@adis:blad.is@adis:blad.isI also managed to solve https://github.com/nix-community/npmlock2nix/issues/86 with some clever setup hook machinery03:49:54
@adis:blad.is@adis:blad.isTook some inspiration from how home-manager deals with unmanaged/managed files03:51:11
@adis:blad.is@adis:blad.ishttps://github.com/adisbladis/buildNodeModules/blob/master/hooks/node-modules-hook.js#L4903:51:29
@adis:blad.is@adis:blad.is So now I can use npm as a locker, and Nix builds everything in node_modules <3 03:51:52
@adis:blad.is@adis:blad.isAnd the shell hook takes care of keeping symlinks up to date03:52:08
@adis:blad.is@adis:blad.isCreated a readme at https://github.com/adisbladis/buildNodeModules04:16:34
@adis:blad.is@adis:blad.is

I converted github-copilot-cli in nixpkgs and here are the eval performance results:

  • before.json
{
  "cpuTime": 0.7983409762382507,
  "envs": {
    "bytes": 16531624,
    "elements": 828921,
    "number": 618766
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 180765008
  },
  "list": {
    "bytes": 3000672,
    "concats": 21613,
    "elements": 375084
  },
  "nrAvoided": 864119,
  "nrFunctionCalls": 558479,
  "nrLookups": 201175,
  "nrOpUpdateValuesCopied": 4886502,
  "nrOpUpdates": 45948,
  "nrPrimOpCalls": 391174,
  "nrThunks": 1152515,
  "sets": {
    "bytes": 98766016,
    "elements": 5996044,
    "number": 176832
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 386981,
    "number": 37425
  },
  "values": {
    "bytes": 36922296,
    "number": 1538429
  }
}
  • after.json
{
  "cpuTime": 0.510466992855072,
  "envs": {
    "bytes": 9614136,
    "elements": 488211,
    "number": 356778
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 91434160
  },
  "list": {
    "bytes": 1416640,
    "concats": 14601,
    "elements": 177080
  },
  "nrAvoided": 511866,
  "nrFunctionCalls": 321155,
  "nrLookups": 126367,
  "nrOpUpdateValuesCopied": 2282699,
  "nrOpUpdates": 28766,
  "nrPrimOpCalls": 240324,
  "nrThunks": 639741,
  "sets": {
    "bytes": 45028896,
    "elements": 2733143,
    "number": 81163
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 370066,
    "number": 34762
  },
  "values": {
    "bytes": 20532984,
    "number": 855541
  }
}
10:01:22
@c0ba1t:matrix.orgCobaltAre the results in any way surprising? The cputime looks a bit too small to be used for reliable measurements without perf counters10:41:33
@adis:blad.is@adis:blad.isNot surprising to me. I was expecting it to be a bit faster and use less memory I'm not overly concerned with exact wall time, I'm OK concluding it's a bit faster and leave it at that.11:11:13
@adis:blad.is@adis:blad.isA concern that often comes up when when talking about lock file parsing in nixpkgs performance. I wanted to have ballpark numbers to refer to. 11:12:52
@c0ba1t:matrix.orgCobaltAh, okay. Thank you for sharing11:13:31
@raitobezarius:matrix.orgraitobezarius
In reply to @adis:blad.is
A concern that often comes up when when talking about lock file parsing in nixpkgs performance.
I wanted to have ballpark numbers to refer to.
You know that I nerdsnipped @tomberek on adding the parsing built-in based on a EBNF grammar (a packrat parser) 		11:48:39
@raitobezarius:matrix.orgraitobezariusHe got OK speed I think11:48:46

Show newer messages

Back to Room ListRoom Version: 6