Move towards using standard kubernetes provider for manifests/crds

[viniciusdc]

Recently I've made a simple test running the QHub deployment on a freshly installed machine. Basically I've set up minikube + kubectl and spun up a minikube cluster. Followed the Local Testing section of the Testing docs and it all worked fine until "Deploy and render the infrastructure" where I got the following error when deploying:

[terraform]: │ Error: Failed to determine GroupVersionResource for manifest
[terraform]: │ 
[terraform]: │   with module.qhub.module.kubernetes-dask-gateway.kubernetes_manifest.main,
[terraform]: │   on modules/kubernetes/services/dask-gateway/crds.tf line 1, in resource "kubernetes_manifest" "main":
[terraform]: │    1: resource "kubernetes_manifest" "main" {
[terraform]: │ 
[terraform]: │ cannot select exact GV from REST mapper

I've tried to search about that problem but didn't find much information regarding terraform usage. Fortunatly, I was able to solve that problem removing the docker engine installation and changing it's version from latest to 20.10.8 as well as directly specifying the kubernetes version for the cluster:
minikube start --kubernetes-version=1.19.4 --driver=docker --cpus 4 --memory 4096

There might be some changes within the latest versions of the necessary components for the local deployment, so I would suggest a review in our docs.

[magsol]

Tried again, changing only the kubernetes version by including the flag --kubernetes-version=v1.19.4 in the minikube start command, as my Docker engine was already at version 20.10.8 (that seems like it's the current latest stable?).

It did require that I spin down and delete my existing minikube cluster, as it was running kubernetes 1.22. As such, I had to back up and repeat some of the previous steps in the Local Testing instructions to initialize the minikube cluster. As mentioned, the only step that changed was including an explicit kubernetes version in the minikube startup command.

The rest of the steps proceeded smoothly and without problems exactly as indicated.

Of course, when attempting to navigate to github-actions.qhub.dev/hub/login in the browser, Chrome nuked the connection due to an overdeveloped cert management system, but that's for a different ticket. curl was able to access the page just fine. Closing.

[costrouc]

Going to reopen this issue seem related to hashicorp/terraform-provider-kubernetes-alpha#235. Looks like terraform now supports a kubernetes_manifest and we should now start to use the kubernetes provider for all this work. https://www.hashicorp.com/blog/beta-support-for-crds-in-the-terraform-provider-for-kubernetes

https://github.com/hashicorp/terraform-provider-kubernetes

[costrouc]

Done in #1003. Closing issue.