Get rid of coredns sidecar for nsc to simplify NSM usage in k8s

[denis-tingaikin]

Context

https://github.com/networkservicemesh/deployments-k8s/tree/release/v1.4.0/examples/features/dns

Description

Currently we're using the coredns side car as a proxy dns server that fanounts dns queries to different dns worlds (in particular k8s dns)

We're using WH to inject this sidecar for nsm clients
https://github.com/networkservicemesh/cmd-admission-webhook-k8s/blob/main/main.go#L226-L242

Problems

1. workload 1/1 is turning to 3/3 when we add nsm annotation in k8s (we could reduce it to 2/2 where 1/2 - workload, 2/2 nsc)
2. we need to add shared volume in pod https://github.com/networkservicemesh/cmd-admission-webhook-k8s/blob/main/main.go#L163-L171
3. coredns costs resouces that is critical for horizontal scaling.
4. coredns reload plugin is too slow (it tooks ~2s to reload configuraiton that could be super critical for complex production cases)
5. we need to deploy/manage fanout plugin

Solution

1. Remove coredns usage
2. Run simple dns server (based on coredns main lib https://github.com/miekg/dns) as a goroutine

Note: if we'll need to get a special Coredns functionally we could simply imprort the plugin and adapt it for our dns server. https://github.com/networkservicemesh/sdk/blob/main/pkg/tools/dnsutils/adapt/handler.go#L17

Potential dns chain elements could be

func main() {
...
    var dnsServerHandler = next.NewDNSHandler(
          resolveconf.NewDNSHandler(),
          dnsconfigs.NewDNSHandler(),
          searches.NewDNSHandler(),
          noloop.NewDNSHandler(),
          norecursion.NewDNSHandler(),
          cache.NewDNSHandler(...),
          fanout.NewDNSHandler(....),
     )
  go dnsutils.ListenAndServe(dnsServerHandler, ...)
...
}

Some of chain elements alerady done see at https://github.com/networkservicemesh/sdk/tree/main/pkg/tools/dnsutils

[denis-tingaikin]

/cc @edwarnicke , @fkautz

[edwarnicke]

@denis-tingaikin I like this approach very much :)

[denis-tingaikin]

TODO

SDK

• Rework fanout dns chain element to force work with context instead of function.
• Rework dnscontext.NetworkServiceClient => create resolvconf dns chain element. That simply edits resolv.conf file by analogous with dnscontext.NetworkServiceClient
• Rework dnscontext.NetworkServiceClient => create new dnsconfigs dns chain element via shared map. That adds into context fanout list and searches.
• create new searches dns chain element that adds to question sub-domains.
• create new cache dns chain element that stores answers for TTL period.

cmd-nsc

• Add new goroutine dns server

cmd-nsc-init

• Add new goroutine dns server

cmd-admission-webhook

• Remove everything that related to Coredns.

Defenition of done

All subtasks are completed https://github.com/networkservicemesh/deployments-k8s/tree/main/examples/features/dns is working without changes.

[denis-tingaikin]

This one is resolved!

@NikitaSkrynnik Thanks!