| 19 Aug 2023 |
 @khalilsantana:matrix.org | Like some sort of "host <x> built this and stored it in <central_server>", then "host <y> fetches from <central_server> before building packages" | 13:20:17 |
| @khalilsantana:matrix.org | Hum, there's a binary cache wiki page | 13:21:55 |
 Wanja Hentze | In reply to @khalilsantana:matrix.org
Like some sort of "host built this and stored it in <central_server>", then "host fetches from <central_server> before building packages" idk I don't use remote builders | 16:22:35 |
| Wanja Hentze | it all builds on the deploying host, not the deployed host | 16:22:46 |
| Wanja Hentze | which is a beefy CI machine in this case | 16:23:07 |
| Wanja Hentze | maybe you're confused because you're used to running nixos-rebuild on the hosts | 16:26:04 |
| Wanja Hentze | colmena by default builds locally and then pushes to the remote hosts | 16:26:17 |
| Wanja Hentze | so in that sense you're automatically sharing built things as long as you're deploying several hosts from the same local host | 16:26:42 |
| @khalilsantana:matrix.org | But I mean, say if I add some new package to my config.nix, that will be built locally (deployment machine) and then the artefacts copied to the remote targets. But can I somehow fetch whatever package upgrades needed for existing hosts, build them once and share them across hosts using colmena (I seems like no?) | 19:16:04 |
| Wanja Hentze | (I don't understand the question but maybe somebody else does) | 19:21:37 |
 Zhaofeng Li | In reply to @khalilsantana:matrix.org
But I mean, say if I add some new package to my config.nix, that will be built locally (deployment machine) and then the artefacts copied to the remote targets. But can I somehow fetch whatever package upgrades needed for existing hosts, build them once and share them across hosts using colmena (I seems like no?) This is what currently happens. When you deploy a new config, Colmena won't copy paths that already exist on the remote host. | 19:24:38 |
| Zhaofeng Li | For example, you edit networking.hostName, the new system profile closure is mostly identical to the old one except for a few paths | 19:25:59 |
| Zhaofeng Li | * For example, when you only edit networking.hostName, the new system profile closure is mostly identical to the old one except for a few paths | 19:26:11 |
| Zhaofeng Li | The same applies to just adding new packages. Or are you asking about something else? | 19:26:55 |
| @khalilsantana:matrix.org | My question is centered around package versions rather than anything else. Say all my hosts have foo-bar-v1.0, then at a later point the nixos-channel provides foo-bar-v2.0, since the config.nix didn't change I don't think colmena will fetch & apply the new binaries to the remote hosts, correct? | 19:29:07 |
| @khalilsantana:matrix.org | Or does a colmena apply also applies package upgrades, not only installs/uninstalls? | 19:29:50 |
| Zhaofeng Li | Nix works fundamentally differently from other package managers. The package versions are decided when you evaluate the expression, and for Colmena everything is evaluated locally on the host you are deploying from | 19:31:33 |
| Zhaofeng Li | so with channels, it will follow whatever you have on the machine running colmena | 19:31:59 |
| @khalilsantana:matrix.org | Pretty neat! So I guess I don't need to do anything special, just run an apply once in a while | 19:33:55 |
| @khalilsantana:matrix.org | (I've seen there's an auto-upgrade option in nixos itself, but I'd rather have more control) | 19:34:29 |
| Zhaofeng Li | As a more concrete demonstration, run nix-build '<nixpkgs>' -A hello and you will get a path. This path is unique to this specific version of hello, tied to everything that it depends on:
$ nix-store -qR /nix/store/qi2wm8z1m8c8x164ki6r4fbpfl7is735-hello-2.12.1
/nix/store/2y9zl8ky5ac28ali6ly1zfz11d4fq48b-xgcc-12.3.0-libgcc
/nix/store/ic9wnagwh22yhqh3lcdlnv5m178w6f0f-libunistring-1.1
/nix/store/a9mxddm4a5p4kp84vys4n2nrmwqgk7kr-libidn2-2.3.4
/nix/store/1x4ijm9r1a88qk7zcmbbfza324gx1aac-glibc-2.37-8
/nix/store/qi2wm8z1m8c8x164ki6r4fbpfl7is735-hello-2.12.1
| 19:37:53 |
| Wanja Hentze | In reply to @khalilsantana:matrix.org
(I've seen there's an auto-upgrade option in nixos itself, but I'd rather have more control) yes, I don't think the auto-upgrade feature will work well at all with colmena | 19:40:23 |
| Zhaofeng Li | Say later there's a vulnerability that requires you to patch glibc-2.37-8. Nothing changes about this specific hello - It will still use the same exact unpatched glibc-2.37-8 as you originally built it. Therefore there aren't really upgrades - Any change results in a totally different package in a different path | 19:41:45 |
| Zhaofeng Li | Sorry if this is too verbose - I wanted to clear some things up, as the questions about "upgrades" and "adding new packages" suggested some misunderstanding | 19:43:04 |
| @khalilsantana:matrix.org | Understood. I like the in-depth explanations since I'm pretty new to nix, so I'm still getting my mind wrapped around it. It certainly doesn't help that the nix pkg in my distro was/is busted | 19:45:57 |
| Zhaofeng Li | The core thing is that system profiles themselves are packages as well and follow the same rules. In other words, when you deploy, you always apply the whole coherent configuration and there's no way to not to fetch upgrades depending on what the host currently has | 19:55:35 |
| Zhaofeng Li | * The core thing is that system profiles themselves are packages as well and follow the same rules. In other words, when you deploy, you always apply the whole coherent configuration and there's no way to not fetch upgrades depending on what the host currently has | 19:55:48 |
| Zhaofeng Li | The older versions of packages are still there in the store, just "dormant" until you garbage collect them | 19:56:37 |
| Wanja Hentze | also, I recommend puttinf something like this in your colmena config for all hosts
```nix
environment.etc."nixos/configuration.nix".text = ''
throw "sorry, no nixos-rebuild, use colmena"
'';
``` | 20:07:50 |
| Wanja Hentze | * also, I recommend puttinf something like this in your colmena config for all hosts
```
environment.etc."nixos/configuration.nix".text = ''
throw "sorry, no nixos-rebuild, use colmena"
'';
``` | 20:08:08 |
