Migrating config entries into custom resources

[lkysow]

Support annotation consul.hashicorp.com/migrate-entry on custom
resources that will allow an existing config entry to be migrated onto
Kubernetes. The config entry from then on will be managed by Kubernetes.

This will support existing users of the Helm chart that already have
config entries in Consul and allow them to migrate to CRDs.

How I've tested this PR:

• code
• manual testing

How I expect reviewers to test this PR:

1. spin up cluster with

   global:
     name: consul
     imageK8S: ghcr.io/lkysow/consul-k8s-dev:jan12
   server:
     replicas: 1
   connectInject:
     enabled: true
   controller:
     enabled: true

2. create a service-defaults config:

   kubectl exec consul-server-0 -- echo '{"kind": "service-defaults", "name": "foo", "protocol": "http"}' | consul config write -

3. read it

   kubectl exec consul-server-0 -- consul config read -kind service-defaults -name foo
   {
       "Kind": "service-defaults",
       "Name": "foo",
       "Protocol": "http",
       "MeshGateway": {},
       "Expose": {},
       "CreateIndex": 973,
       "ModifyIndex": 973
   }

4. create a matching CR:

   cat <<EOF | kubectl apply -f -
   apiVersion: consul.hashicorp.com/v1alpha1
   kind: ServiceDefaults
   metadata:
     name: foo
   spec:
     protocol: "http"
   EOF

5. check that it's not syncing:

   kubectl describe servicedefaults foo
   ...
   Status:
     Conditions:
       Last Transition Time:  2021-01-12T21:03:29Z
       Message:               config entry managed in different datacenter: ""
       Reason:                ExternallyManagedConfigError
       Status:                False
       Type:                  Synced

6. add the annotation:

   cat <<EOF | kubectl apply -f -
   apiVersion: consul.hashicorp.com/v1alpha1
   kind: ServiceDefaults
   metadata:
     name: foo
     annotations:
       "consul.hashicorp.com/migrate-entry": "true"
   spec:
     protocol: "http"
   EOF

7. check that it's synced:

   kubectl describe servicedefaults foo
   ...
   Status:
     Conditions:
       Last Transition Time:  2021-01-12T21:04:37Z
       Status:                True
       Type:                  Synced

8. check it's got the expected metadata

   kubectl exec consul-server-0 -- consul config read -kind service-defaults -name foo
   {
       "Kind": "service-defaults",
       "Name": "foo",
       "Protocol": "http",
       "MeshGateway": {},
       "Expose": {},
       "Meta": {
           "consul.hashicorp.com/source-datacenter": "dc1",
           "external-source": "kubernetes"
       },
       "CreateIndex": 973,
       "ModifyIndex": 1065
   }

Checklist:

• Tests added
• CHANGELOG entry added (HashiCorp engineers only, community PRs should not add a changelog entry)

[kschoche]

excellent work!

[ndhanushkodi]

Looks great to me, thanks for the detailed testing steps, they worked!

[lkysow]

The test failure is a flake fixed in #421.