| 14 Sep 2022 |
 @palo:xaos.space | Hey, I'm using colmena to provison a machine (in another continent) I set deployment.buildOnTarget = true and ran colmena apply -v and I see that it's copying a lot of dervations (2399 paths) from my local machine to the target machine. Among these derivations are things like
91wqgl41iwq10kp3kp5v9386xsjd3hw2-texinfo-6.8.drv
jk6nxrnvrr8kzwnfbazsa204hj0acdb2-linux-headers-5.19.drv
v031hl8l2yk3drnw1bc950x9k4wiak0n-texlive-grfext-1.3.drv
Is there a good way to avoid this? Meaning is there a way to only upload the most minimal set of derivations needed? | 04:14:26 |
| @palo:xaos.space | * Hey, I'm using colmena to provison a machine (in another continent) I set deployment.buildOnTarget = true and ran colmena apply -v and I see that it's copying a lot of dervations (2399 paths) from my local machine to the target machine. Among these derivations are things like
- 91wqgl41iwq10kp3kp5v9386xsjd3hw2-texinfo-6.8.drv
- jk6nxrnvrr8kzwnfbazsa204hj0acdb2-linux-headers-5.19.drv
- v031hl8l2yk3drnw1bc950x9k4wiak0n-texlive-grfext-1.3.drv
Is there a good way to avoid this? Meaning is there a way to only upload the most minimal set of derivations needed?
| 04:15:14 |
| @palo:xaos.space | * Hey, I'm using colmena to provison a machine (in another continent) I set deployment.buildOnTarget = true and ran colmena apply -v and I see that it's copying a lot of dervations (2399 paths) from my local machine to the target machine. Among these derivations are things like
- 91wqgl41iwq10kp3kp5v9386xsjd3hw2-texinfo-6.8.drv
- jk6nxrnvrr8kzwnfbazsa204hj0acdb2-linux-headers-5.19.drv
- v031hl8l2yk3drnw1bc950x9k4wiak0n-texlive-grfext-1.3.drv
Is there a good way to avoid this? Meaning is there a way to only upload the most minimal set of derivations needed? | 04:15:27 |
 Zhaofeng Li | Currently this isn't possible before we have remote evaluation, but you can try to minimize the config itself by disabling documentations for example. | 06:09:57 |
| @palo:xaos.space | In reply to @zhaofeng:zhaofeng.li
Currently this isn't possible before we have remote evaluation, but you can try to minimize the config itself by disabling documentations for example. Ok, disabled all documentation parts I know of, but maybe I missed something. It's a price for comfort I have to pay I guess :D | 08:24:26 |
 @yuka:yuka.dev | sounds like the problem here is that nix-copy-closure is highly latency dependent when copying many small paths? | 08:38:29 |
| @yuka:yuka.dev | it could be worked around by exporting the entire .drv closure, compressing it, and sending it over | 08:38:45 |
| @yuka:yuka.dev | nix-store --export $(nix-store -qR ...)
| zstd
| ssh remote "zstd -d | sudo nix-store --import"
| 08:40:11 |
 Linux Hackerman | Yureka (she/her): recent versions of nix aren't as latency-sensitive in nix copy anymore -- as long as you use ssh-ng and not ssh | 08:42:42 |
| Linux Hackerman | so nix copy --to ssh-ng://user@host --derivation /nix/store/...-foo.drv should be pretty fast | 08:43:07 |
| @yuka:yuka.dev | But colmena uses ssh:// not ssh-ng://? | 08:43:31 |
| Linux Hackerman | palo: ^ if you copy the drv using the fast method first, colmena won't have much (anything) left to copy | 08:43:41 |
| Linux Hackerman | Yureka (she/her): I actually have no idea. I suspect that it does, because of using nix-copy-closure (which definitely does use ssh://), but I haven't looked at any code to confirm that. | 08:44:18 |
 Wanja Hentze | what in the fresh hell is ssh-ng? | 08:45:19 |
| @yuka:yuka.dev | next gen :p | 08:45:28 |
| Wanja Hentze | I figured as much | 08:45:38 |
| @yuka:yuka.dev | proj/nix/src/libstore/legacy-ssh-store.cc
proj/nix/src/libstore/ssh-store.cc
| 08:47:03 |
| Linux Hackerman | it's a newer way for nix to operate, using nix-daemon --stdio instead of nix-store --serve iirc | 08:47:05 |
| Linux Hackerman | it's a different and more capable (though still not good :p ) protocol | 08:47:24 |
| Linux Hackerman | the same one that's also used to communicate with the local daemon | 08:47:33 |
| Wanja Hentze | o nice | 08:52:05 |
| @palo:xaos.space | How to find out the derivativation that is generated by colmena, so I can upload it with ssh-ng ? | 09:42:11 |
| Linux Hackerman | uuuuh good question | 09:48:15 |
| Linux Hackerman | In reply to @yuka:yuka.dev
nixosConfigurations = (import (inputs.colmena + "/src/nix/hive/eval.nix") {
rawFlake = self;
colmenaOptions = import (inputs.colmena + "/src/nix/hive/options.nix");
colmenaModules = import (inputs.colmena + "/src/nix/hive/modules.nix");
}).nodes;
palo: if you're using flakes you can add something like this and nix eval .#nixosConfigurations.$hostname.config.system.toplevel.drvPath | 09:51:24 |
| @palo:xaos.space | ah nice, thanks, I'll try that | 09:56:13 |
| @palo:xaos.space | but I'm just using outputs.colmena.$hostname.config.system.toplevel.drvPath in my case. I haven't seen this colmenaOptions and colmenaModules so far. | 09:58:13 |
| @palo:xaos.space | * but I'm just using outputs.colmena.$hostname.config.system.toplevel.drvPath in my case I guess. I haven't seen this colmenaOptions and colmenaModules so far. | 09:59:25 |
| @palo:xaos.space | Hmm, I can't make it work :( | 10:03:03 |
| @yuka:yuka.dev | In reply to @palo:xaos.space
but I'm just using outputs.colmena.$hostname.config.system.toplevel.drvPath in my case I guess. I haven't seen this colmenaOptions and colmenaModules so far. that won't work | 10:07:12 |
| @yuka:yuka.dev | tbh proves my point about colmena magically injecting stuff through its own entry point is a bad idea, it is very hard to understand for people how to actually evaluate your system outside of colmena then | 10:08:13 |
