| 15 Apr 2024 |
 hexa | still no luck with ceph 😕 | 16:15:21 |
| hexa | # incus launch images:nixos/unstable test
Launching test
Error: Failed instance creation: Failed creating instance from image: Failed to run: rbd --id admin --cluster ceph --pool incus_rbd map image_2eadc6e7ec07d26a23a72fe092f79369317fce83f04b2179e00e6c5216cb2e0a_ext4: exit status 110 (rbd: sysfs write failed
rbd: map failed: (110) Connection timed out)
| 16:15:45 |
| hexa | incus created that pool | 16:16:00 |
| hexa | but mounting a block device from it fails | 16:16:11 |
| hexa | [265181.093176] libceph: mon2 (1)[2001:41b8:41:41:9618:82ff:fe08:ea7c]:6789 session established
[265181.093815] libceph: another match of type 1 in addrvec
[265181.094280] libceph: corrupt full osdmap (-22) epoch 256 off 1066 (00000000f270e95d of 0000000079bf1b0b-000000003a874c07)
[...]
| 16:16:32 |
 adamcstephens | oof | 16:22:17 |
| adamcstephens | did you ever get ovn working? | 16:25:48 |
| hexa | I got it clustering | 16:35:06 |
| hexa | but unless I have storage there is no reason to toy with networking | 16:35:17 |
| adamcstephens | right | 16:56:13 |
| 16 Apr 2024 |
| hexa | ok | 12:11:18 |
| hexa | killed ipv4 on the ceph cluster, reoboted, tada | 12:11:24 |
| hexa | 
Download image.png | 12:13:57 |
| adamcstephens | so ipv4 was doing something weird? | 12:58:33 |
| hexa | 0.0.0.0 addresses in the osdmap, multiples actually | 13:00:43 |
| hexa | and that confused the kernel's libceph driver | 13:00:56 |
| 17 Apr 2024 |
| hexa | I hot-added CPU cores to a VM … wow 😄 | 09:39:08 |
| hexa | htop shows them as offlien thought, so I wonder if our nixos image is missing anything | 09:39:19 |
| hexa | oofff | 10:11:50 |
| hexa | # incus move test --target incus2.karo.tu.da.man-da.net
Error: Migration operation failure: Instance move to destination failed: Error transferring instance data: Failed migration on target: Failed restoring checkpoint from source: Monitor is disconnected
root@incus1.karo.tu.da:~# incus list
+------+---------+------+------+-----------------+-----------+------------------------------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | LOCATION |
+------+---------+------+------+-----------------+-----------+------------------------------+
| test | STOPPED | | | VIRTUAL-MACHINE | 0 | incus1.karo.tu.da.man-da.net |
+------+---------+------+------+-----------------+-----------+------------------------------+
| 10:11:57 |
| hexa | wow | 10:14:59 |
| hexa | # incus move test --target incus2.karo.tu.da.man-da.net
Error: Migration operation failure: Instance move to destination failed on source: Failed migration on source: Failed stopping instance: Failed unmounting instance: Failed to run: rbd --id admin --cluster ceph --pool incus_rbd unmap virtual-machine_test: exit status 16 (rbd: sysfs write failed
rbd: unmap failed: (16) Device or resource busy)
root@incus1.karo.tu.da:~# incus list
+------+---------+------+------+-----------------+-----------+------------------------------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | LOCATION |
+------+---------+------+------+-----------------+-----------+------------------------------+
| test | STOPPED | | | VIRTUAL-MACHINE | 0 | incus1.karo.tu.da.man-da.net |
+------+---------+------+------+-----------------+-----------+------------------------------+
| 10:15:02 |
| hexa | but the VM is actually up | 10:15:35 |
| hexa | and was migrated to incus2 | 10:15:56 |
| hexa | 🫠| 10:15:58 |
| adamcstephens | Yeah we may be missing CPU hotplug | 11:02:30 |
| hexa | I'll file an issue for that as well | 11:15:42 |
| adamcstephens | I suspect that one is on our end | 11:25:17 |
| hexa | Yep | 11:35:11 |
| adamcstephens | Probably an easy one to add a test for too :) | 12:31:05 |
