| 27 May 2024 |
 hexa | ok, probably have to look into zfs as the backing storage now as well | 08:37:31 |
 adamcstephens | i've zfs for years with incus/lxd, even moving between systems and pools. no problems i can remember | 12:48:04 |
| hexa | so it does zfs send for stateful migration I assume? | 12:50:09 |
| hexa | and ultimately an instance has its storage only backed by whatever raid constellation the local machine it currently resides on, correct? | 12:51:03 |
| adamcstephens | yeah i think it uses zfs send for a migration | 13:36:25 |
| adamcstephens | i've only tried that once or twice | 13:36:34 |
| adamcstephens | ceph not working? | 13:36:48 |
| hexa | so how does it catch up with changes to the zvol, since the guest is still running | 13:36:56 |
| hexa | I can get it to work, I'm not sure I can reasonably get the colleagues to a point where they're in a position to maintain and debug it 🙂 | 13:37:32 |
| hexa | we're primarily a network carrier | 13:37:52 |
| adamcstephens | ❯ incus move blank:just-bobcat bonk:
Error: Failed instance creation: Expected "criu" connection secret missing from migration sink target request
| 13:43:06 |
| adamcstephens | hmm | 13:43:07 |
| hexa | container? | 13:45:01 |
| adamcstephens | no, it's a VM | 13:45:43 |
| hexa | huh, so why CRIU | 13:46:13 |
| hexa | or is that stateful migration with ZFS? 😄 | 13:46:35 |
| hexa | * or is that a stateful migration with ZFS? 😄 | 13:46:39 |
| hexa | where it tries to freeze the VM | 13:46:47 |
| adamcstephens | yeah thats a stateful migration | 13:47:00 |
| adamcstephens | a stateless migration works, so maybe we're missing something | 13:52:51 |
| hexa | does your kernel have CRIU? | 13:56:20 |
| hexa | $ zgrep -i CHECKPOINT_RESTORE /proc/config.gz
CONFIG_CHECKPOINT_RESTORE=y
| 13:57:55 |
| hexa | kernel 5.9. added CAP_CHECKPOINT_RESTORE | 13:58:05 |
| adamcstephens | yeah i'm using the nixos 6.6 lts | 13:58:22 |
| hexa | yeah, just if you have set a capab boundingset | 13:58:41 |
| adamcstephens | ❯ /nix/store/d5f5grj684mp1xl6h7llgr2cklpdg2z4-criu-3.19/bin/criu check
CRIU needs to have the CAP_SYS_ADMIN or the CAP_CHECKPOINT_RESTORE capability:
setcap cap_checkpoint_restore+eip /nix/store/d5f5grj684mp1xl6h7llgr2cklpdg2z4-criu-3.19/bin/criu
~ adam@blank
❯ sudo /nix/store/d5f5grj684mp1xl6h7llgr2cklpdg2z4-criu-3.19/bin/criu check
Looks good.
~ adam@blank
❯ sudo systemd-analyze security incus | rg sys_admin
✗ CapabilityBoundingSet=~CAP_SYS_ADMIN Service has administrator privileges
| 14:01:31 |
| hexa | and the "or" is not misleading? | 14:13:23 |
| adamcstephens | incus runs as root with admin capability. is that not sufficient? | 14:23:59 |
| hexa | the messages implies that | 14:31:50 |
| hexa | just want to rule out we're missing anything obvious due to that sentence being dogshit | 14:32:10 |
