!djTaTBQyWEPRQxrPTb:nixos.org

Nixpkgs Architecture Team

232 Members
https://github.com/nixpkgs-architecture, weekly public meetings on Wednesday 15:00-16:00 UTC at https://meet.jit.si/nixpkgs-architecture52 Servers

Load older messages


SenderMessageTime
21 Jul 2022
@infinisil:matrix.orginfinisilMaybe let's have a when2meet.com14:29:04
@Ericson2314:matrix.orgJohn EricsonI can do right now if you want14:29:54
@Ericson2314:matrix.orgJohn Ericsonor when2meet14:29:56
@infinisil:matrix.orginfinisilOh neat, yeah let's do it now then14:30:26
@infinisil:matrix.orginfinisilI'm in https://meet.jit.si/nixpkgs-architecture14:31:26
@infinisil:matrix.orginfinisil Alyssa Ross: Wanna join too? 14:32:38
@infinisil:matrix.orginfinisilGonna take notes in https://pad.lassul.us/axm8GR1_RKmJnqd0UOjHFQ?edit14:33:58
@qyliss:fairydust.spaceAlyssa Rossinfinisil: is it still going?15:04:34
@infinisil:matrix.orginfinisil Alyssa Ross: Yes 15:05:10
@infinisil:matrix.orginfinisil John Ericson: Are you interested in joining the team officially? 20:27:23
@Ericson2314:matrix.orgJohn EricsonSure!20:29:49
@infinisil:matrix.orginfinisilI pushed the meeting notes from todays off-hour meeting to https://github.com/nixpkgs-architecture/meetings btw20:32:41
@infinisil:matrix.orginfinisil John Ericson: Cool! For the formals, hit me with your request to join as described in that section 20:49:39
@Ericson2314:matrix.orgJohn EricsonOK 20:49:59
@armeen:matrix.orgarmeen joined the room.22:15:13
22 Jul 2022
@asrar_:matrix.orgasrar joined the room.19:23:05
@claesatwork:matrix.orgClaes at work joined the room.19:30:33
23 Jul 2022
@growpotkin1:matrix.orggrowpotkin1 joined the room.05:52:43
@growpotkin1:matrix.orggrowpotkin1

I was sent here after asking for a meeting with the fixed-point/overlay grey-beards.

I'm 3-4 months down a design rabbit hole and I think it's about time to come talk to the folks who have shot themselves in the foot a few times before I injure myself lol.

I'll just include my original post:

If you are or know who are the wizards who wrote some of the Haskell newScope or related overlays - I would like to formally request counsel with the elders.
I've been working on some open source Nix+Node.js tooling for ~3 months, and I've accomplished quite a bit; but I'm at this stage where I'm going off the deep end with fixes/extends to basically create crawlers for package metadata. It's a blast; but I need to consult with people who have shot their feet a few times with this stuff because it can turn into a real time sink and I'm going to be out of toes soon. Haskell here is just an example because it is abundantly clear that those authors also went - fully off the deep end; but honestly if you've built any of the fixed-points or overlay systems in nixpkgs let me know.
My puzzle for today has basically been "when should I stack up raw overlays, and at which breakpoints in a pipeline do I compose and apply them". I sort arrived at the realization that a break between "gather pure metadata", optionally "gather impure metadata", "generate stage 1", and "get wild with a scoped callPackage style final round" are sensible; but deciding which of those are exposed for users is and experimenting more is getting nasty because I'm so deep down the path on certain patterns. I need a guide.
I went from "keep it simple stupid" a month ago to "absolutely every piece of data needs to be overridable" once I started collecting all of my functions/builders into an API and a part of me sort of knows "I have loaded a large caliber footgun that is going to go off soon" lol.

05:58:16
@growpotkin1:matrix.orggrowpotkin1Overall, I'm exaggerating for effect here; but I have definitely spent two weeks playing with overlays/fixed-points and as education as it has been I need to spend some time in Office Hours to get some direction lol06:00:19
@growpotkin1:matrix.orggrowpotkin1 * Overall, I'm exaggerating for effect here, I've honestly got a lot done and could crank this out as something "functional" but somewhat inflexible in a day or two. But inflexible ain't gonna cut it - I'm trying to make a tool we can all actually live with! I'm close, but I have definitely spent two weeks playing with overlays/fixed-points and as education as it has been I need to spend some time in Office Hours to get some direction lol06:02:32
@growpotkin1:matrix.orggrowpotkin1 *

I was sent here after seeking the wisdom of "the fixed-point/overlay grey-beards".

I'm 3-4 months down a design rabbit hole and I think it's about time to come talk to the folks who have shot themselves in the foot a few times before I injure myself lol.

I'll just include my original post:

If you are or know who are the wizards who wrote some of the Haskell newScope or related overlays - I would like to formally request counsel with the elders.
I've been working on some open source Nix+Node.js tooling for ~3 months, and I've accomplished quite a bit; but I'm at this stage where I'm going off the deep end with fixes/extends to basically create crawlers for package metadata. It's a blast; but I need to consult with people who have shot their feet a few times with this stuff because it can turn into a real time sink and I'm going to be out of toes soon. Haskell here is just an example because it is abundantly clear that those authors also went - fully off the deep end; but honestly if you've built any of the fixed-points or overlay systems in nixpkgs let me know.
My puzzle for today has basically been "when should I stack up raw overlays, and at which breakpoints in a pipeline do I compose and apply them". I sort arrived at the realization that a break between "gather pure metadata", optionally "gather impure metadata", "generate stage 1", and "get wild with a scoped callPackage style final round" are sensible; but deciding which of those are exposed for users is and experimenting more is getting nasty because I'm so deep down the path on certain patterns. I need a guide.
I went from "keep it simple stupid" a month ago to "absolutely every piece of data needs to be overridable" once I started collecting all of my functions/builders into an API and a part of me sort of knows "I have loaded a large caliber footgun that is going to go off soon" lol.

06:05:06
@growpotkin1:matrix.orggrowpotkin1 *

I was sent here after seeking the wisdom of "the fixed-point/overlay grey-beards".

I'm 3-4 months down a design rabbit hole and I think it's about time to come talk to the folks who have shot themselves in the foot a few times before I injure myself. I'm not a JS user, honestly don't particularly like web stuff; but work asked me to "make Nix+Node tooling that the an intern could handle - you can open source it, ~6 month timeline". I'm close but I know myself and I've been playing with "fun" fixed points for like 2 weeks and if one of you don't save me from myself... 😅

I'll just include my original post.

If you are or know who are the wizards who wrote some of the Haskell newScope or related overlays - I would like to formally request counsel with the elders.
I've been working on some open source Nix+Node.js tooling for ~3 months, and I've accomplished quite a bit; but I'm at this stage where I'm going off the deep end with fixes/extends to basically create crawlers for package metadata. It's a blast; but I need to consult with people who have shot their feet a few times with this stuff because it can turn into a real time sink and I'm going to be out of toes soon. Haskell here is just an example because it is abundantly clear that those authors also went - fully off the deep end; but honestly if you've built any of the fixed-points or overlay systems in nixpkgs let me know.
My puzzle for today has basically been "when should I stack up raw overlays, and at which breakpoints in a pipeline do I compose and apply them". I sort arrived at the realization that a break between "gather pure metadata", optionally "gather impure metadata", "generate stage 1", and "get wild with a scoped callPackage style final round" are sensible; but deciding which of those are exposed for users is and experimenting more is getting nasty because I'm so deep down the path on certain patterns. I need a guide.
I went from "keep it simple stupid" a month ago to "absolutely every piece of data needs to be overridable" once I started collecting all of my functions/builders into an API and a part of me sort of knows "I have loaded a large caliber footgun that is going to go off soon" lol.

06:13:41
@growpotkin1:matrix.orggrowpotkin1 * Overall, I'm exaggerating for effect here, I've honestly got a lot done and could crank this out as something "functional" but somewhat inflexible in a day or two. But inflexible ain't gonna cut it - I'm trying to make a tool we can all actually live with! ( "we" as in "us Nix nerds" - because I'm hoping that if the intern understands it, it'll be useful in Nixpkgs ). I'm close, but I have definitely spent two weeks playing with overlays/fixed-points and as education as it has been I need to spend some time in Office Hours to get some direction lol06:24:13
@growpotkin1:matrix.orggrowpotkin1And just to sweeten the pot here: what I have written so far beats both Yarn and NPM on speed and uses less than half of the system resources - with a bare Nix store. With a populated store it's no contest, and I'm obviously an excentric but I think there's a real possibility that if this was flexible Nix could be a legitimate competitor and pick up a chunk of new users. Nix roped me in ~6 years ago by blowing Cabal and Stack out of the water and JS could be an even bigger draw. And just to clear the air a bit : I was upfront about the fact that there's overlap with work/OSS here but I maintain `libtool` for GNU and you can rest assured that I ain't fishing for anybody to consult on this just to turn it over to the company. 06:38:13
@growpotkin1:matrix.orggrowpotkin1* And just to sweeten the pot here: what I have written so far beats both Yarn and NPM on speed and uses less than half of the system resources - with a bare Nix store. With a populated store it's no contest. I'm obviously an excentric but I think there's a real possibility that if this was flexible Nix could be a legitimate competitor and pick up a chunk of new users. Nix roped me in ~6 years ago by blowing Cabal and Stack out of the water and JS could be an even bigger draw. And just to clear the air a bit : I was upfront about the fact that there's overlap with work/OSS here but I maintain `libtool` for GNU and you can rest assured that I ain't fishing for anybody to consult on this just to turn it over to the company. 06:40:05
@growpotkin1:matrix.orggrowpotkin1* And just to sweeten the pot here: what I have written so far beats both Yarn and NPM on speed and uses less than half of the system resources - with a bare Nix store. With a populated store it's no contest. I'm obviously an excentric but I think there's a real possibility that if this was flexible and user ( intern ) friendly, Nix could be a legitimate competitor and pick up a chunk of new users. Nix roped me in ~6 years ago by blowing Cabal and Stack out of the water and JS could be an even bigger draw. And just to clear the air a bit : I was upfront about the fact that there's overlap with work/OSS here but I maintain `libtool` for GNU and you can rest assured that I ain't fishing for anybody to consult on this just to turn it over to the company. 06:40:50
@growpotkin1:matrix.orggrowpotkin1The Nixpkgs' manual for Node.js currently recommends using Nix as a wrapper around NPM/Yarn. I disagreed, and mapped `package.json` and locks directly into derivations - this approach never invokes NPM or Yarn. I spent months reimplementing those tools as pure Nix expressions, and I gated all IFD behind conditionals so it could be used in Nixpkgs. This, converts and optimizes NPM and Yarn projects into Nix projects, and while it's seamless, it leads users to further optimize their build by using Nix directly. 06:48:04
@growpotkin1:matrix.orggrowpotkin1* The Nixpkgs' manual for Node.js currently recommends using Nix as a wrapper around NPM/Yarn. I disagreed, and mapped `package.json` and locks directly into derivations - this approach never invokes NPM or Yarn. I spent months reimplementing those tools as pure Nix expressions, and I gated all IFD behind conditionals so it could be used in Nixpkgs. This, converts and optimizes NPM and Yarn projects into Nix projects, and while it's seamless, it leads users to further optimize their build by using Nix directly. If you want to use it on top on your slower package manager you can; but if you're trying to sell your org on Nix this flow let's you infect during a transition and "flip the switch" to pure Nix when you get the green light. This isn't incidental. It's exactly what I needed to do at my org, and I imagine that it's something most of y'all have schemed over at some point 😂06:55:12
@profpatsch:augsburg.oneprofpatschholy hell your employer is paying for all of this?14:13:05

Show newer messages


Back to Room ListRoom Version: 9