| 10 Jun 2021 |
 rager | hi - anybody dealt with networking with the in-tree kubernetes service? | 23:24:37 |
| 11 Jun 2021 |
 Mic92 (Old) | @rager: sort of | 07:54:47 |
|  kraynyan joined the room. | 09:35:47 |
|  zarel joined the room. | 10:51:30 |
|  rushmore joined the room. | 14:12:13 |
| 12 Jun 2021 |
| rager | was trying to use nixos as a router but add k8s | 16:41:40 |
| rager | at least the native nat setup can't route into services, just pods - is a problem of ordering of iptables rules? | 16:42:27 |
| rager | configuration.network.nat | 16:42:58 |
|  keithy joined the room. | 17:24:53 |
| keithy | I have just upgraded to 21.05 and on reboot network-setup is failing with network-setup-start[1654]: Error: Nexthop has invalid gateway. | 17:25:53 |
| keithy | It seems to work when I manually start it | 17:26:05 |
| keithy | I'm a bit puzzled | 17:26:32 |
| keithy | wondered if anyone has any ideas | 17:26:42 |
|  tnias joined the room. | 17:32:31 |
| Mic92 (Old) | In reply to @rager:synapse.lickmy.app
was trying to use nixos as a router but add k8s I also disabled the firewall. You need to whitelist all incoming ports at least like the api server. | 17:38:21 |
| rager | since it's also my router, I'm going to stick with the firewall for now - problem seems to be that by the time the rules run that forward nat traffic, the tables that route k8s services seem to have already been evaluated | 17:40:49 |
| Mic92 (Old) | I rage quitted debugging k8s firewall rules. They go beyond my understanding :) | 17:41:39 |
| rager | I'm not far from there | 17:42:03 |
| Mic92 (Old) | I guess that's why people just put k8s in another container | 17:42:04 |
| rager | put it in a VM, and I can see that making sense | 17:42:17 |
| rager | else, it's all the same kernel | 17:42:23 |
| rager | or is namespace enough to make the rules happen when you "want" them to? | 17:43:12 |
| Mic92 (Old) | I now how to write network drivers or extend systemd-networkd but I don't understand k8s firewall rules :) | 17:43:24 |
| Mic92 (Old) | In reply to @rager:synapse.lickmy.app
or is namespace enough to make the rules happen when you "want" them to? yes, a network namespace should be sufficient. | 17:43:42 |
| rager | I think my iptables issue comes down to these two snippets:
-N nixos-nat-pre
-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A PREROUTING -j nixos-nat-pre
and
-A nixos-nat-pre -i eno1 -p tcp -m tcp --dport 6666 -j DNAT --to-destination 10.10.142.1:22
-A nixos-nat-pre -i eno1 -j DNAT --to-destination 10.0.0.75
(context: https://hastebin.com/ijusozofeb.yaml)
| 22:49:47 |
| rager | though I'm not sure what happens after a packet gets dnat'd to an ip that corresponds to an device on the same host | 22:50:33 |
| rager | because I'm real bad at iptables | 22:50:44 |
 casey © | the thing i missed most going from a bsd universe to linux, lack of pf. | 23:14:56 |
| rager | ok... I got it to work | 23:38:48 |
| rager | step 1: don't configure anything from nixos any more | 23:39:03 |
