!UKDpaKNNsBpOPfLWfX:zhaofeng.li

Colmena

311 Members
A simple, stateless NixOS deployment tool - https://github.com/zhaofengli/colmena110 Servers

Load older messages


SenderMessageTime
16 Aug 2022
@ruddy:leohoo.xyzruddy left the room.12:12:22
@rdk31:matrix.orgrdk31 joined the room.19:39:44
@kayla.fire:matrix.orgkayla (she/they) joined the room.21:30:57
17 Aug 2022
@zhaofeng:zhaofeng.liZhaofeng Li
In reply to @blaggacao:matrix.org

Well, colmena uses an evaluator (instead of for example the nix bundle interface) in order to parallelize the evaluation. So at its heart, it's a speed consideración.

The receptor interface is established in src/nix/eval.nix and that defines the incoming schema as well as the intermediate schema that can be processed by the CLI.

There are two solutions:

  • don't use an evaluator, but rather consume a well-knowm flake attribute (which is already evaluated - this is the divnix/std approach with __std)
  • make the evaluator swappable colmena --eval ./path/to/eval.nix so that you can consume arbitrary inout contracts via a custom evaluator.

Since colmena strives for optimization and parallelization, the second option is really the beat choixe.

For something like divnix/styx, I don't know yet, I'll probably just going to wrap the nix bundle interface within the CLI. It's probably not parallelized, but it is easly swappable for a custom evaluator. WIP, so I might get more insight as I go.

Ok, finally have some time for Colmena, so let's talk about swappable eval. My primary concern with it is that it may limit the potential for larger changes to the deployment flow, such as support for something like profiles/activators in deploy-rs. I'm also a bit wary of wrapped versions of Colmena with the swappable evaluator overridden exhibiting totally different behavior, causing confusions with the user.
03:03:38
@zhaofeng:zhaofeng.liZhaofeng Li

A new related development is that flake evaluation is finally pure [1]. The current implementation is to generate a temporary flake containing eval.nix and friends that depends on the user flake as an input.

[1] For nix-instantiate at the moment. The streaming evaluator nix-eval-jobs requires a change to be upstreamed.

03:03:56
@winterqt:nixos.devWinter (she/her) Has anyone noticed their building with buildOnTarget get slower recently, or is this just me/my network? Even with an existing copy (build then apply), the copying 0 paths... takes way longer than it has in the past/should. 05:03:59
@winterqt:nixos.devWinter (she/her)Not sure how I'd debug this, though.05:04:20
@winterqt:nixos.devWinter (she/her)* Not sure how I'd debug this, though, as it's most likely a Nix issue or a networking issue... idk.05:44:39
@blaggacao:matrix.orgDavid Arnold (blaggacao)

An eval is a transformer that enriches one data-structure with something else to emit a different datastructure that the CLI then consumes. In theory, there is nothing in the evaler that one couldn't also do directly in the target flake.

So it offloads the implementation of an interface onto the tooling when it is considered closer to the tooling domain in order to reduce boilerplate or enforce order on the origin data structure.

That enforcing of a certain order on the origin datastructure is contentions because it forces convergence outside of the tooling domain (where it was precisely deemed more appropriate). This time: not conformance to the target data structure, but to the expected input interface.

19:23:22
@blaggacao:matrix.orgDavid Arnold (blaggacao) Actually, similar to my styx approach, I belive that Colmena, instead of working with a wrapper flake, could also make most benefit of the nix bundle interface which doesn precisely that job, is pure, and if I'm not ill-imformed, even has (or could have in future) specialized cross-flake caching support (unlike anything else). 19:25:33
@blaggacao:matrix.orgDavid Arnold (blaggacao)* Actually, similar to my `styx` approach, I belive that Colmena, instead of working with a wrapper flake, could also make most benefit of the `nix bundle` interface which does precisely that job, is pure, and if I'm not ill-imformed, even has (or could have in future) specialized cross-flake caching support (unlike anything else).19:25:52
@blaggacao:matrix.orgDavid Arnold (blaggacao)The beauty, the bundler approach is inherently very swappable, since it would only replace one bundler with a fork of that same bundler.19:26:49
@blaggacao:matrix.orgDavid Arnold (blaggacao)

Now, input schema enforcement is on the bundler (which is fine and absolutely what we want anyways).

Output schema enforcement can be placed on the bundler (and can be even provided by a flake input coming from Colmena).

So when a bundler updates the colmena output schema reference, then it would also emit appropriate errors if that new schema isn't satisfied by a target flake.

Vice-versa, the cli can do a preflight check on the bundler (nix eval targetbundled#colemna-version) and error out if compatibility cannot be guaranteed.

19:30:25
@blaggacao:matrix.orgDavid Arnold (blaggacao)That way, either colmena CLI or the bundler itself error out on schema violations19:30:47
@blaggacao:matrix.orgDavid Arnold (blaggacao)* That way, either colmena CLI or the bundler itself error out on output schema violations19:31:00
@greaka:greaka.degreaka ⚡️ joined the room.20:23:07
18 Aug 2022
@tpw_rules:matrix.orgtpw_rulesis the nixpkgs in the colmena meta used for all thehosts?19:03:38
@tpw_rules:matrix.orgtpw_rules i'd really like to map nixosConfigurations in my flake to the colmena output but that doesn't quite seem possible 19:05:49
@phaer:matrix.orgphaer
In reply to @tpw_rules:matrix.org
is the nixpkgs in the colmena meta used for all thehosts?
Did you see this already? https://github.com/zhaofengli/colmena/issues/60#issuecomment-1047199551
21:01:19
19 Aug 2022
@kayla.fire:matrix.orgkayla (she/they) changed their display name from kayla.fire to kayla (she/they).01:39:55
@tpw_rules:matrix.orgtpw_rulesi did now, thank you. it's still unclear what the meta nixpkgs is for?03:28:58
@winterqt:nixos.devWinter (she/her)
In reply to @tpw_rules:matrix.org
is the nixpkgs in the colmena meta used for all thehosts?
Yes, see https://colmena.cli.rs/unstable/reference/meta.html#nixpkgs
04:58:33
@winterqt:nixos.devWinter (she/her)Also see the one below it, which allows you to set a Nixpkgs pin per node.04:58:54
@winterqt:nixos.devWinter (she/her)* Also see the option below it, which allows you to set a Nixpkgs pin per node.04:59:00
20 Aug 2022
@hanemile:matrix.orghanemile changed their profile picture.15:56:46
@janejasperous:one.ems.hostJane Jasperous where i need to declare system to keep running colmena without --impure flag? 17:39:31
@janejasperous:one.ems.hostJane Jasperous * where i need to declare system to keep running colmena 0.4.0-pre without --impure flag? 17:40:09
21 Aug 2022
@winterqt:nixos.devWinter (she/her)
In reply to @janejasperous:one.ems.host
where i need to declare system to keep running colmena 0.4.0-pre without --impure flag?
meta.nixpkgs = import nixpkgs { system = "..."; }
01:43:59
@winterqt:nixos.devWinter (she/her)see https://colmena.cli.rs/unstable/tutorial/flakes.html#basic-configuration01:44:20
@janejasperous:one.ems.hostJane Jasperous
In reply to @winterqt:nixos.dev
meta.nixpkgs = import nixpkgs { system = "..."; }
is not working for me then
08:40:09

There are no newer messages yet.


Back to Room ListRoom Version: 6