!UKDpaKNNsBpOPfLWfX:zhaofeng.li

Colmena

329 Members
A simple, stateless NixOS deployment tool - https://github.com/zhaofengli/colmena116 Servers

Load older messages


SenderMessageTime
13 Aug 2022
@farcaller:matrix.orgfarcallergotcha, thanks for clarifying17:27:47
@jhillyerd:matrix.orgjhillyerdI think the old channel configs hang around, so you can end up with some very old environments when using nix-env or nix-shell on those hosts. Not sure if there is a better way to control that.18:12:35
@farcaller:matrix.orgfarcallerefficiency wise that means I can’t quite build a base image and finish the configuration with colmena because that will effectively overwrite everything and negate any benefits of deploying from a snapshot, right?18:24:44
@jhillyerd:matrix.orgjhillyerdIf the base and the colmena config are built off of different versions of nixpkgs, then yes, you'd be uploading mostly new /nix/store contents18:25:59
@farcaller:matrix.orgfarcalleroh right; and I can freeze those.18:28:31
@farcaller:matrix.orgfarcallerthanks!18:28:35
15 Aug 2022
@tim92:matrix.orgtim joined the room.06:10:23
16 Aug 2022
@ruddy:leohoo.xyz@ruddy:leohoo.xyz joined the room.12:12:19
@ruddy:leohoo.xyz@ruddy:leohoo.xyz left the room.12:12:22
@rdk31:matrix.orgrdk31 joined the room.19:39:44
@kayla.fire:matrix.orgkayla (she/they) joined the room.21:30:57
17 Aug 2022
@zhaofeng:zhaofeng.liZhaofeng Li
In reply to @blaggacao:matrix.org

Well, colmena uses an evaluator (instead of for example the nix bundle interface) in order to parallelize the evaluation. So at its heart, it's a speed consideración.

The receptor interface is established in src/nix/eval.nix and that defines the incoming schema as well as the intermediate schema that can be processed by the CLI.

There are two solutions:

  • don't use an evaluator, but rather consume a well-knowm flake attribute (which is already evaluated - this is the divnix/std approach with __std)
  • make the evaluator swappable colmena --eval ./path/to/eval.nix so that you can consume arbitrary inout contracts via a custom evaluator.

Since colmena strives for optimization and parallelization, the second option is really the beat choixe.

For something like divnix/styx, I don't know yet, I'll probably just going to wrap the nix bundle interface within the CLI. It's probably not parallelized, but it is easly swappable for a custom evaluator. WIP, so I might get more insight as I go.

Ok, finally have some time for Colmena, so let's talk about swappable eval. My primary concern with it is that it may limit the potential for larger changes to the deployment flow, such as support for something like profiles/activators in deploy-rs. I'm also a bit wary of wrapped versions of Colmena with the swappable evaluator overridden exhibiting totally different behavior, causing confusions with the user.
03:03:38
@zhaofeng:zhaofeng.liZhaofeng Li

A new related development is that flake evaluation is finally pure [1]. The current implementation is to generate a temporary flake containing eval.nix and friends that depends on the user flake as an input.

[1] For nix-instantiate at the moment. The streaming evaluator nix-eval-jobs requires a change to be upstreamed.

03:03:56
@winterqt:nixos.devWinter (she/her) Has anyone noticed their building with buildOnTarget get slower recently, or is this just me/my network? Even with an existing copy (build then apply), the copying 0 paths... takes way longer than it has in the past/should. 05:03:59
@winterqt:nixos.devWinter (she/her)Not sure how I'd debug this, though.05:04:20
@winterqt:nixos.devWinter (she/her)* Not sure how I'd debug this, though, as it's most likely a Nix issue or a networking issue... idk.05:44:39
@blaggacao:matrix.orgDavid Arnold (blaggacao)

An eval is a transformer that enriches one data-structure with something else to emit a different datastructure that the CLI then consumes. In theory, there is nothing in the evaler that one couldn't also do directly in the target flake.

So it offloads the implementation of an interface onto the tooling when it is considered closer to the tooling domain in order to reduce boilerplate or enforce order on the origin data structure.

That enforcing of a certain order on the origin datastructure is contentions because it forces convergence outside of the tooling domain (where it was precisely deemed more appropriate). This time: not conformance to the target data structure, but to the expected input interface.

19:23:22
@blaggacao:matrix.orgDavid Arnold (blaggacao) Actually, similar to my styx approach, I belive that Colmena, instead of working with a wrapper flake, could also make most benefit of the nix bundle interface which doesn precisely that job, is pure, and if I'm not ill-imformed, even has (or could have in future) specialized cross-flake caching support (unlike anything else). 19:25:33
@blaggacao:matrix.orgDavid Arnold (blaggacao)* Actually, similar to my `styx` approach, I belive that Colmena, instead of working with a wrapper flake, could also make most benefit of the `nix bundle` interface which does precisely that job, is pure, and if I'm not ill-imformed, even has (or could have in future) specialized cross-flake caching support (unlike anything else).19:25:52
@blaggacao:matrix.orgDavid Arnold (blaggacao)The beauty, the bundler approach is inherently very swappable, since it would only replace one bundler with a fork of that same bundler.19:26:49
@blaggacao:matrix.orgDavid Arnold (blaggacao)

Now, input schema enforcement is on the bundler (which is fine and absolutely what we want anyways).

Output schema enforcement can be placed on the bundler (and can be even provided by a flake input coming from Colmena).

So when a bundler updates the colmena output schema reference, then it would also emit appropriate errors if that new schema isn't satisfied by a target flake.

Vice-versa, the cli can do a preflight check on the bundler (nix eval targetbundled#colemna-version) and error out if compatibility cannot be guaranteed.

19:30:25
@blaggacao:matrix.orgDavid Arnold (blaggacao)That way, either colmena CLI or the bundler itself error out on schema violations19:30:47
@blaggacao:matrix.orgDavid Arnold (blaggacao)* That way, either colmena CLI or the bundler itself error out on output schema violations19:31:00
@greaka:greaka.degreaka joined the room.20:23:07
18 Aug 2022
@tpw_rules:matrix.orgtpw_rulesis the nixpkgs in the colmena meta used for all thehosts?19:03:38
@tpw_rules:matrix.orgtpw_rules i'd really like to map nixosConfigurations in my flake to the colmena output but that doesn't quite seem possible 19:05:49
@phaer:matrix.orgphaer
In reply to @tpw_rules:matrix.org
is the nixpkgs in the colmena meta used for all thehosts?
Did you see this already? https://github.com/zhaofengli/colmena/issues/60#issuecomment-1047199551
21:01:19
19 Aug 2022
@kayla.fire:matrix.orgkayla (she/they) changed their display name from kayla.fire to kayla (she/they).01:39:55
@tpw_rules:matrix.orgtpw_rulesi did now, thank you. it's still unclear what the meta nixpkgs is for?03:28:58
@winterqt:nixos.devWinter (she/her)
In reply to @tpw_rules:matrix.org
is the nixpkgs in the colmena meta used for all thehosts?
Yes, see https://colmena.cli.rs/unstable/reference/meta.html#nixpkgs
04:58:33

Show newer messages


Back to Room ListRoom Version: 6