!djTaTBQyWEPRQxrPTb:nixos.org

Nixpkgs Architecture Team

222 Members
https://github.com/nixpkgs-architecture, weekly public meetings on Wednesday 15:00-16:00 UTC at https://meet.jit.si/nixpkgs-architecture51 Servers

Load older messages


SenderMessageTime
6 Dec 2023
@infinisil:matrix.orginfinisil Philip Taron: Oh awesome, that would be very welcome, thanks! I'll mark it as ready to review when done :) 05:12:01
12 Dec 2023
@infinisil:matrix.orginfinisilNothing concrete, but just wanted to point out this issue: https://github.com/NixOS/nixpkgs/issues/27353417:43:46
@tomberek:matrix.orgtomberek
In reply to @infinisil:matrix.org
Nothing concrete, but just wanted to point out this issue: https://github.com/NixOS/nixpkgs/issues/273534
We keep seeing this. And the current flake usage makes this worse. I opened: https://github.com/nixpkgs-architecture/issues/issues/25 , still needs work and discussion.
17:49:48
@roberthensing:matrix.orgRobert Hensing (roberth)Braindump of an actually feasible module-like packaging method for Nixpkgs, if you're interested. https://github.com/NixOS/nixpkgs/issues/27381519:46:52
@bew:matrix.orgbewFound an unfinished sentence starting with "This should not include redundant (and slow)" 22:29:32
@roberthensing:matrix.orgRobert Hensing (roberth)Thanks. Edited. That was going to be about mkDerivation package attr stuff and multi-outputs, but the latter got its own section.23:06:06
13 Dec 2023
@nbp:mozilla.orgnbp“Bring all packaging layers into a single fixpoint” This is already the case, unless you are thinking about the bootstrap of the compilers, which is something I wanted to at one point.10:09:14
@nbp:mozilla.orgnbp Nixpkgs is already a fix-point of merged functions (final: prev: { … }) 10:13:53
@nbp:mozilla.orgnbp Robert Hensing (roberth): I think we disagree on what qualifies as a module system, and your definition already encapsulate Nixpkgs as a module system, as I understand it. 10:23:11
@nbp:mozilla.orgnbp

The core of the problem of override is that we have to peel the layers of executed functions in order to patch what was inside.

This can be avoided by not calling mkDerivation but making it an attribute in the attribute set which is used to define a package, and then add an extra phase in charge of evaluating packages by calling the attribute which holds the mkDerivation with the attribute set of the package. I was not able to see this extra phase in your brain dump.

10:26:48
@nbp:mozilla.orgnbp And if you follow all these minimal set of steps, you end-up with S.O.S. which comes with the idea that we need a better way of merging values together which does not involve repeating tons of // operators, just to override something within an attribute set. 10:29:13
@roberthensing:matrix.orgRobert Hensing (roberth) What I describe merely takes a concept from the module system and uses it on its own. I don't think I've defined anything. 12:02:09
@roberthensing:matrix.orgRobert Hensing (roberth) My proposal is smaller in scope, keeping mkDerivation-like calls in top-level and such. It only tries to solve problems within a single package. 12:04:58
@roberthensing:matrix.orgRobert Hensing (roberth)My proposal solves the problem of too many layers of abstraction, not the problem of override memory consumption or anything related to top-level. Nonetheless it may unify how packages are constructed, having less diverse calls in top-level etc, so it could be a stepping stone towards S.O.S.12:11:59
@nbp:mozilla.orgnbpOh … I completely misunderstood the proposal :/ Which is about using the module system -like to update the content of a derivation, and not to update Nixpkgs. Sorry, I would have to re-read based on this expectation.14:38:13
@roberthensing:matrix.orgRobert Hensing (roberth)Don't worry. We don't normally talk about layers at the individual package level, so I can see why you associated it that way. (Also the proposal is to remove the ad hoc and excessive layering, so that word is kind of on the way out I guess - just useful for describing the current situation and the transition)15:09:49
@nbp:mozilla.orgnbp one idea of light weight merging which is almost what S.O.S. suggested would be to have a zipWithRightMergeFun, if the right hand side is a function, then this is considered to be a merging function.
Then replacing is _: newValue, appending is old: old + "exit 0", debugging is x: __trace x x
15:19:03
@nbp:mozilla.orgnbpThe problem is that this is per-value, and thus we lose the recurisve attribute sets, where one define the result based on another attribute. This could still be provided through the final resume-point, assuming that the resolver is at the end of Nixpkgs.15:21:13
@roberthensing:matrix.orgRobert Hensing (roberth)The exact merging semantics is to be decided. It will be easier when we have a prototype that we can benchmark and play around with.15:36:58
@roberthensing:matrix.orgRobert Hensing (roberth)Added that to the issue text as well15:38:14
@roberthensing:matrix.orgRobert Hensing (roberth) * Added that to the issue text as well just now15:38:21
@infinisil:matrix.orginfinisil nbp: That actually reminds me of https://nixos.org/manual/nixpkgs/stable/#function-library-lib.attrsets.updateManyAttrsByPath 16:05:14
@9999years:matrix.org9999years joined the room.20:23:53
@9999years:matrix.org9999years

should RFC 140 be a mechanism just for nixpkgs (which has fairly unique requirements, like being large enough to necessitate sharding package names) or should it also be a user-facing mechanism for creating package sets from directories?

i'd like to add a user-facing mechanism for creating package sets from directories (#270537) and infinisil hasn't been sure whether or not it should be the same mechanism as pkgs/by-name. i'd like to make a decision on this so that we can ship something, and then we can iterate and improve from there

20:27:16
@infinisil:matrix.orginfinisil 9999years: A big future idea with RFC 140 was to use the package directories for more than just the package definition itself. 23:47:57
@infinisil:matrix.orginfinisil So we could have e.g. a module.nix file that specifies a NixOS module (or home-manager module, or some abstract module, ..) 23:48:21
@infinisil:matrix.orginfinisil Or a meta.nix, which would allow querying package metadata without much evaluation 23:48:39
@infinisil:matrix.orginfinisil Or some way to specify overrides. args.nix maybe, but that's very ad-hoc. Maybe rather something like dependencies.nix and config.nix or so 23:49:34
@infinisil:matrix.orginfinisil And this kind of idea would be really useful to have standardised, such that people can write third-party package directories, and upstreaming to Nixpkgs will be just a matter of a cp -r. Or even better: Allow third-party repositories to have their package directories be automatically pulled into Nixpkgs 23:50:38
@infinisil:matrix.orginfinisilAnd yes this does sound a lot like the Flakes output schema, but I don't think we can use that directly in its current state23:52:56

Show newer messages


Back to Room ListRoom Version: 9