Nixpkgs Architecture Team - Public Room Timeline

	Nixpkgs Architecture Team	235 Members
	https://github.com/nixpkgs-architecture, weekly public meetings on Wednesday 15:00-16:00 UTC at https://meet.jit.si/nixpkgs-architecture	52 Servers

Load older messages

Sender	Message	Time
21 Jul 2022
John Ericson	Sure!	20:29:49
infinisil	I pushed the meeting notes from todays off-hour meeting to https://github.com/nixpkgs-architecture/meetings btw	20:32:41
infinisil	John Ericson: Cool! For the formals, hit me with your request to join as described in that section	20:49:39
John Ericson	OK	20:49:59
	armeen joined the room.	22:15:13
22 Jul 2022
	asrar joined the room.	19:23:05
	Claes at work joined the room.	19:30:33
23 Jul 2022
	growpotkin1 joined the room.	05:52:43
growpotkin1	I was sent here after asking for a meeting with the fixed-point/overlay grey-beards. I'm 3-4 months down a design rabbit hole and I think it's about time to come talk to the folks who have shot themselves in the foot a few times before I injure myself lol. I'll just include my original post: If you are or know who are the wizards who wrote some of the Haskell newScope or related overlays - I would like to formally request counsel with the elders. I've been working on some open source Nix+Node.js tooling for ~3 months, and I've accomplished quite a bit; but I'm at this stage where I'm going off the deep end with fixes/extends to basically create crawlers for package metadata. It's a blast; but I need to consult with people who have shot their feet a few times with this stuff because it can turn into a real time sink and I'm going to be out of toes soon. Haskell here is just an example because it is abundantly clear that those authors also went - fully off the deep end; but honestly if you've built any of the fixed-points or overlay systems in nixpkgs let me know. My puzzle for today has basically been "when should I stack up raw overlays, and at which breakpoints in a pipeline do I compose and apply them". I sort arrived at the realization that a break between "gather pure metadata", optionally "gather impure metadata", "generate stage 1", and "get wild with a scoped callPackage style final round" are sensible; but deciding which of those are exposed for users is and experimenting more is getting nasty because I'm so deep down the path on certain patterns. I need a guide. I went from "keep it simple stupid" a month ago to "absolutely every piece of data needs to be overridable" once I started collecting all of my functions/builders into an API and a part of me sort of knows "I have loaded a large caliber footgun that is going to go off soon" lol.	05:58:16
growpotkin1	Overall, I'm exaggerating for effect here; but I have definitely spent two weeks playing with overlays/fixed-points and as education as it has been I need to spend some time in Office Hours to get some direction lol	06:00:19
growpotkin1	* Overall, I'm exaggerating for effect here, I've honestly got a lot done and could crank this out as something "functional" but somewhat inflexible in a day or two. But inflexible ain't gonna cut it - I'm trying to make a tool we can all actually live with! I'm close, but I have definitely spent two weeks playing with overlays/fixed-points and as education as it has been I need to spend some time in Office Hours to get some direction lol	06:02:32
growpotkin1	* I was sent here after seeking the wisdom of "the fixed-point/overlay grey-beards". I'm 3-4 months down a design rabbit hole and I think it's about time to come talk to the folks who have shot themselves in the foot a few times before I injure myself lol. I'll just include my original post: If you are or know who are the wizards who wrote some of the Haskell newScope or related overlays - I would like to formally request counsel with the elders. I've been working on some open source Nix+Node.js tooling for ~3 months, and I've accomplished quite a bit; but I'm at this stage where I'm going off the deep end with fixes/extends to basically create crawlers for package metadata. It's a blast; but I need to consult with people who have shot their feet a few times with this stuff because it can turn into a real time sink and I'm going to be out of toes soon. Haskell here is just an example because it is abundantly clear that those authors also went - fully off the deep end; but honestly if you've built any of the fixed-points or overlay systems in nixpkgs let me know. My puzzle for today has basically been "when should I stack up raw overlays, and at which breakpoints in a pipeline do I compose and apply them". I sort arrived at the realization that a break between "gather pure metadata", optionally "gather impure metadata", "generate stage 1", and "get wild with a scoped callPackage style final round" are sensible; but deciding which of those are exposed for users is and experimenting more is getting nasty because I'm so deep down the path on certain patterns. I need a guide. I went from "keep it simple stupid" a month ago to "absolutely every piece of data needs to be overridable" once I started collecting all of my functions/builders into an API and a part of me sort of knows "I have loaded a large caliber footgun that is going to go off soon" lol.	06:05:06
growpotkin1	* I was sent here after seeking the wisdom of "the fixed-point/overlay grey-beards". I'm 3-4 months down a design rabbit hole and I think it's about time to come talk to the folks who have shot themselves in the foot a few times before I injure myself. I'm not a JS user, honestly don't particularly like web stuff; but work asked me to "make Nix+Node tooling that the an intern could handle - you can open source it, ~6 month timeline". I'm close but I know myself and I've been playing with "fun" fixed points for like 2 weeks and if one of you don't save me from myself... 😅 I'll just include my original post. If you are or know who are the wizards who wrote some of the Haskell newScope or related overlays - I would like to formally request counsel with the elders. I've been working on some open source Nix+Node.js tooling for ~3 months, and I've accomplished quite a bit; but I'm at this stage where I'm going off the deep end with fixes/extends to basically create crawlers for package metadata. It's a blast; but I need to consult with people who have shot their feet a few times with this stuff because it can turn into a real time sink and I'm going to be out of toes soon. Haskell here is just an example because it is abundantly clear that those authors also went - fully off the deep end; but honestly if you've built any of the fixed-points or overlay systems in nixpkgs let me know. My puzzle for today has basically been "when should I stack up raw overlays, and at which breakpoints in a pipeline do I compose and apply them". I sort arrived at the realization that a break between "gather pure metadata", optionally "gather impure metadata", "generate stage 1", and "get wild with a scoped callPackage style final round" are sensible; but deciding which of those are exposed for users is and experimenting more is getting nasty because I'm so deep down the path on certain patterns. I need a guide. I went from "keep it simple stupid" a month ago to "absolutely every piece of data needs to be overridable" once I started collecting all of my functions/builders into an API and a part of me sort of knows "I have loaded a large caliber footgun that is going to go off soon" lol.	06:13:41
growpotkin1	* Overall, I'm exaggerating for effect here, I've honestly got a lot done and could crank this out as something "functional" but somewhat inflexible in a day or two. But inflexible ain't gonna cut it - I'm trying to make a tool we can all actually live with! ( "we" as in "us Nix nerds" - because I'm hoping that if the intern understands it, it'll be useful in Nixpkgs ). I'm close, but I have definitely spent two weeks playing with overlays/fixed-points and as education as it has been I need to spend some time in Office Hours to get some direction lol	06:24:13
growpotkin1	And just to sweeten the pot here: what I have written so far beats both Yarn and NPM on speed and uses less than half of the system resources - with a bare Nix store. With a populated store it's no contest, and I'm obviously an excentric but I think there's a real possibility that if this was flexible Nix could be a legitimate competitor and pick up a chunk of new users. Nix roped me in ~6 years ago by blowing Cabal and Stack out of the water and JS could be an even bigger draw. And just to clear the air a bit : I was upfront about the fact that there's overlap with work/OSS here but I maintain `libtool` for GNU and you can rest assured that I ain't fishing for anybody to consult on this just to turn it over to the company.	06:38:13
growpotkin1	* And just to sweeten the pot here: what I have written so far beats both Yarn and NPM on speed and uses less than half of the system resources - with a bare Nix store. With a populated store it's no contest. I'm obviously an excentric but I think there's a real possibility that if this was flexible Nix could be a legitimate competitor and pick up a chunk of new users. Nix roped me in ~6 years ago by blowing Cabal and Stack out of the water and JS could be an even bigger draw. And just to clear the air a bit : I was upfront about the fact that there's overlap with work/OSS here but I maintain `libtool` for GNU and you can rest assured that I ain't fishing for anybody to consult on this just to turn it over to the company.	06:40:05
growpotkin1	* And just to sweeten the pot here: what I have written so far beats both Yarn and NPM on speed and uses less than half of the system resources - with a bare Nix store. With a populated store it's no contest. I'm obviously an excentric but I think there's a real possibility that if this was flexible and user ( intern ) friendly, Nix could be a legitimate competitor and pick up a chunk of new users. Nix roped me in ~6 years ago by blowing Cabal and Stack out of the water and JS could be an even bigger draw. And just to clear the air a bit : I was upfront about the fact that there's overlap with work/OSS here but I maintain `libtool` for GNU and you can rest assured that I ain't fishing for anybody to consult on this just to turn it over to the company.	06:40:50
growpotkin1	The Nixpkgs' manual for Node.js currently recommends using Nix as a wrapper around NPM/Yarn. I disagreed, and mapped `package.json` and locks directly into derivations - this approach never invokes NPM or Yarn. I spent months reimplementing those tools as pure Nix expressions, and I gated all IFD behind conditionals so it could be used in Nixpkgs. This, converts and optimizes NPM and Yarn projects into Nix projects, and while it's seamless, it leads users to further optimize their build by using Nix directly.	06:48:04
growpotkin1	* The Nixpkgs' manual for Node.js currently recommends using Nix as a wrapper around NPM/Yarn. I disagreed, and mapped `package.json` and locks directly into derivations - this approach never invokes NPM or Yarn. I spent months reimplementing those tools as pure Nix expressions, and I gated all IFD behind conditionals so it could be used in Nixpkgs. This, converts and optimizes NPM and Yarn projects into Nix projects, and while it's seamless, it leads users to further optimize their build by using Nix directly. If you want to use it on top on your slower package manager you can; but if you're trying to sell your org on Nix this flow let's you infect during a transition and "flip the switch" to pure Nix when you get the green light. This isn't incidental. It's exactly what I needed to do at my org, and I imagine that it's something most of y'all have schemed over at some point 😂	06:55:12
profpatsch	holy hell your employer is paying for all of this?	14:13:05
profpatsch	Sorry I just skimmed the text because the IRC bridge really does not like edits and just duplicates the whole message every time	14:14:01
profpatsch	My unfortunate realization after working with overlays a bunch: It’s not worth the efforts trying to override things in a generalized way, best just focus on the specific task at hand and don’t be afraid to copy & paste some definitions/modules	14:15:31
profpatsch	In particular, a tool that solves all use-cases is not feasible	14:15:48
profpatsch	Plus, if this is for work, maintainability will trump any kind of cleverness, so if you already hit a dead end after going into the deep end for two weeks, think about the poor soul that will take over the codebase when you inevitably leave the company	14:16:49
profpatsch	I saw a good quote yesterday “If you think your code is clever, chances are you are doing something stupid” :P	14:17:30
profpatsch	If by “metadata generation” you mean something like “generate license compliance documents”, that definitely feasible with nixpkgs, but I wouldn’t trust the metadata to be correct. In that case you have to actually unpack the source tarballs and search for the license in there.	14:19:08
profpatsch	I think a good 80/20 solution for that could be possible and would be valuable (even as a product), but no idea whether that’s the use-case.	14:19:50
profpatsch	(plus the tool that has the heuristics for finding license headers etc. is probably more work & value)	14:20:33
growpotkin1	Thanks this is what I needed to hear. Yeah what I before the overlays worked fine and was simple, in cases where you do you need to override you basically had to do it statically in specific files; but that's probably easier to grokk. Maintainability is a good point aa well. I know deep down these slick overlays are going to be hard for contributors to keep clean.	17:07:59
growpotkin1	And yeah I'll admit I got a sweet gig here haha. Pay is a bit lower than average, but they're cool with me running off to play with Nix for six months without pressing for deliverables 😁	17:11:34

Show newer messages

Back to Room ListRoom Version: 9