Support conformance tests w/ minikube

[bobcatfish]

Expected Behavior

As pointed out by @grantr current conformance tests probably do not support minikube due to the need to use the node ip and node port directly when running with minikube.

Actual Behavior

Tests can either:

• Use the domain the controller was configured with (--resolvabledomain)
• Expect the corresponding ingress to be assigned an ip and use that

Steps to Reproduce the Problem

1. Start a minikube cluster that meets the conformance test requirements (elafros, pizzaplanet namespace, docker repo with the required images (e.g. gcr))
2. Run conformance tests - will probably either fail on:
   a. "Make a request to the Revision that is now deployed and serving traffic"
   b. "The Revision will be updated when it is ready to serve traffic" <-- if the controller isn't handling the case where the IP isn't assigned to the ingress

Additional Info

An unknown in this is how we can implement automated testing for this scenario.

[bobcatfish]

A small bug has snuck into the conformance tests that would have to be resolved for this to work - the ingress IP should not be used at all in the --resolvableDomain case - and also in can case added/modified to support minikube

[evankanderson]

/kind feature

Is this still under consideration?

[bobcatfish]

Is this still under consideration?

This one for reals is under construction (unlike #608).

[bobcatfish]

Looks like I won't be getting to this after all, but @jessiezcc + co will be!

[jcrossley3]

I have both e2e and conformance tests passing via go test ... on minikube in a branch based on #1792. The tests pass for two reasons:

1. I'm tagging the test images to avoid the default pull policy of Always for latest tags, though I think I've done it in a way that won't conflict with the latest images pushed by e2e-tests.sh
2. I'm using a hack to make minikube provision LoadBalancer type services

The hack amounts to these two commands:

sudo ip route add $(cat ~/.minikube/profiles/minikube/config.json | jq -r ".KubernetesConfig.ServiceCIDR") via $(minikube ip)
kubectl run minikube-lb-patch --replicas=1 --image=elsonrodriguez/minikube-lb-patch:0.1 --namespace=kube-system

As odious as those might be, I would argue that they simplify the knative-on-minikube user experience considerably:

• no special s/LoadBalancer/NodePort/ install instructions
• no special minikube-only way of obtaining the $SERVICE_IP in the samples
• no need to configure a docker registry in minikube
• and most importantly... all the tests pass.

And since minikube considers its lack of LoadBalancer support to be a bug anyway, it's hopefully only a temporary hack.

I've begun work on the docs in my branch already, but I wanted to get some feedback before documenting the samples, which are in a different repo.