Wrote this up as part of a draft version of the NetBird Kubernetes Access Platform post, but didn't quite fit anymore but I wanted to leave it in for future reference if I ever need it again.
The previous version of the NetBird operator used the cluster DNS domain to map network resources to Kubernetes services. Brief description of that, on the cluster side the operator deploys a set of routing peers which clients form peer-to-peer connections to. On the NetBird side, a Network is populated with all annotated services within the cluster. On the client side, NetBird exposes cluster services as local DNS records, for example nginx.default.svc.cluster.local. GETing this has the client resolve the record, which in turn has one of the routing peer resolve the record through CoreDNS, which returns the resolved ClusterIP address to the client, which then again uses the peer-to-peer tunnel to one of the routing peers for the actual HTTP connection.
So two clusters ending in .svc.cluster.local would have required manually toggling visible networks on the client side. Bad UX, but I didn't want to commit to changing the cluster DNS, so I solved it by adding a small patch to my CoreDNS config:
barracuda.local:53 {
errors
cache 30
kubernetes barracuda.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
}
This keeps both the .svc.cluster.local intact while also allowing each service to be addressed as .svc.barracuda.local. For the Kube API server specifically I also added kubernetes.default.svc.barracuda.local as a cert SAN to be able to address it directly through the NetBird routing peers.