Build Success - but not really with minikube for quickstarts etc

[stevef1uk]

Summary

Unable to build a quick start

Steps to reproduce the behavior

jx create quickstart

I tried the Go & node options as well as the documented spring example. All have the same sort of problem for me.

Expected behavior

Application built and running in Dev

Actual behavior

Build Log implies it is successful but no application running

Jx version

NAME VERSION
jx 2.0.2
jenkins x platform 0.0.3877
Kubernetes cluster v1.14.0
kubectl v1.14.0
helm client Client: v2.13.1+g618447c
git git version 2.21.0
Operating System Mac OS X 10.14.4 build 18E226

Jenkins type

Serverless Jenkins

Kubernetes cluster

Minkube (version minikube version: v1.0.0)
jx create cluster minikube --verbose=true --log-level='info'
I gave it 20Mb of RAM & 7 CPUs to prevent resource starvation & used HyperV

Worth mentioning that my install was struck at this point:
INFO[0345] Waiting for helm post-upgrade hook Job expose to complete before removing it

I resolved that my using the command 'minikube addons enable ingress'

Operating system / Environment

MacOS 18.5.0 Darwin Kernel Version 18.5.0: Mon Mar 11 20:40:32 PDT 2019; root:xnu-4903.251.3~3/RELEASE_X86_64 x86_64

jx env
dev

Last part of Build Log:
INFO[0004] getting the log for build stevef1uk/testing123/master #1 stage from build pack and container build-step-source-mkdir-stevef1uk-testing123-master-mlt4b
{"level":"warn","ts":1555430703.0567784,"logger":"fallback-logger","caller":"logging/config.go:65","msg":"Fetch GitHub commit ID from kodata failed: open /var/run/ko/HEAD: no such file or directory"}
{"level":"info","ts":1555430703.0576735,"logger":"fallback-logger","caller":"bash/main.go:65","msg":"Successfully executed command "mkdir -p /pvc/from-build-pack/workspace""}
INFO[0004] getting the log for build stevef1uk/testing123/master #1 stage from build pack and container build-step-source-copy-stevef1uk-testing123-master-wxv9q
{"level":"warn","ts":1555430704.0996094,"logger":"fallback-logger","caller":"logging/config.go:65","msg":"Fetch GitHub commit ID from kodata failed: open /var/run/ko/HEAD: no such file or directory"}
{"level":"info","ts":1555430704.105594,"logger":"fallback-logger","caller":"bash/main.go:65","msg":"Successfully executed command "cp -r /workspace/source/. /pvc/from-build-pack/workspace""}

jx get activity -f testing123 -w
Version: 0.0.1
from build pack 2m33s 2m25s Succeeded
Credential Initializer 9tn44 2m33s 0s Succeeded
Git Source Stevef1uk Testing123 Master Dzzr2 2m32s 1s Succeeded https://github.com/stevef1uk/testing123
Place Tools 2m30s 0s Succeeded
Git Merge 2m29s 0s Succeeded
Setup Jx Git Credentials 1m44s 0s Succeeded
Build Npm Install 1m43s 1s Succeeded
Build Npm Test 1m42s 0s Succeeded
Build Container Build 1m41s 9s Succeeded
Build Post Build 1m32s 1s Succeeded
Promote Changelog 1m31s 5s Succeeded
Promote Helm Release 1m25s 3s Succeeded
Promote Jx Promote 1m21s 1m11s Succeeded
Source Mkdir Stevef1uk Testing123 Master Mlt4b 9s 0s Succeeded
Source Copy Stevef1uk Testing123 Master Wxv9q 8s 0s Succeeded
Promote: staging 1m16s 1m6s Succeeded
PullRequest 1m16s 1m5s Succeeded PullRequest: https://github.com/stevef1uk/environment-gemlunar-staging/pull/2 Merge SHA: 095419374992a698002e2bcf770c3ed38f2673a7
Update 11s 1s Succeeded

However:
jx get applications
INFO[0000] No applications found in environments staging, production

jx get app
No Apps found

jx diagnose
Kubernetes PVCs:
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
jenkins-x-chartmuseum Bound pvc-cb4bd4c7-605a-11e9-a845-3a609c73a38d 8Gi RWO standard 64m
jenkins-x-docker-registry Bound pvc-cb845437-605a-11e9-a845-3a609c73a38d 100Gi RWO standard 64m
jenkins-x-mongodb Bound pvc-cbba1579-605a-11e9-a845-3a609c73a38d 8Gi RWO standard 64m
jenkins-x-nexus Bound pvc-cbe04604-605a-11e9-a845-3a609c73a38d 8Gi RWO standard 64m
stevef1uk-jxspring1-master-1-pvc Bound pvc-37355192-605c-11e9-a845-3a609c73a38d 5Gi RWO standard 54m
stevef1uk-testing123-master-1-pvc Bound pvc-0e5e54c9-6061-11e9-a845-3a609c73a38d 5Gi RWO standard 19m
INFO[0006]
Kubernetes Pods:
NAME READY STATUS RESTARTS AGE
crier-5f4d789484-mhdsg 1/1 Running 0 64m
deck-55d99dcc8-86gzf 1/1 Running 0 64m
deck-55d99dcc8-dnzpb 1/1 Running 0 64m
hook-5d76598b45-4hl9j 1/1 Running 0 64m
hook-5d76598b45-vk947 1/1 Running 0 64m
horologium-9579fb97c-fxknz 1/1 Running 0 64m
jenkins-x-chartmuseum-d87cbb789-hq4x4 1/1 Running 0 64m
jenkins-x-controllerbuild-69f6fd94b-fx559 1/1 Running 0 64m
jenkins-x-controllercommitstatus-66bb4599b6-6qv8c 1/1 Running 0 64m
jenkins-x-controllerrole-b688c6f49-gs22s 1/1 Running 0 64m
jenkins-x-controllerteam-6bd95c8d78-h65pn 1/1 Running 0 64m
jenkins-x-controllerworkflow-75fdd4b464-2nr6m 1/1 Running 0 64m
jenkins-x-docker-registry-69d666d455-f749s 1/1 Running 0 64m
jenkins-x-gcactivities-1555428600-p9znn 0/1 Completed 0 51m
jenkins-x-gcpods-1555428600-52ggs 0/1 Completed 0 51m
jenkins-x-heapster-ff6df6848-4dgn2 2/2 Running 0 57m
jenkins-x-mongodb-69f857d6c7-2qzvc 1/1 Running 1 64m
jenkins-x-monocular-api-568769577c-qb7hp 1/1 Running 1 64m
jenkins-x-monocular-prerender-9d99d8f59-kg5hn 1/1 Running 0 64m
jenkins-x-monocular-ui-7c659545b4-fqnfb 1/1 Running 0 64m
jenkins-x-nexus-6bc788447f-7mxpx 1/1 Running 0 64m
pipeline-dd8d4b7c8-fj2fr 1/1 Running 0 64m
pipelinerunner-66c4bdc8f9-qmjkg 1/1 Running 0 64m
plank-57c9b986f9-t2phw 1/1 Running 0 64m
prow-build-cdc9995cb-mfr5b 1/1 Running 0 64m
sinker-6765946596-gcnrd 1/1 Running 0 64m
tekton-pipelines-controller-f49866cd9-fzj2k 1/1 Running 0 64m
tekton-pipelines-webhook-5cb4499fd8-wjrfx 1/1 Running 0 64m
tide-6846f664b7-r2v7m 1/1 Running 0 64m
INFO[0007]
Kubernetes Ingresses:
NAME HOSTS ADDRESS PORTS AGE
chartmuseum chartmuseum.jx.192.168.64.52.nip.io 192.168.64.52 80 58m
deck deck.jx.192.168.64.52.nip.io 192.168.64.52 80 58m
docker-registry docker-registry.jx.192.168.64.52.nip.io 192.168.64.52 80 58m
hook hook.jx.192.168.64.52.nip.io 192.168.64.52 80 58m
monocular monocular.jx.192.168.64.52.nip.io 192.168.64.52 80 58m
nexus nexus.jx.192.168.64.52.nip.io 192.168.64.52 80 58m
tide tide.jx.192.168.64.52.nip.io 192.168.64.52 80 58m
INFO[0007]
Kubernetes Secrets:
NAME TYPE DATA AGE
cleanup-token-sklfm kubernetes.io/service-account-token 3 64m
crier-token-q8v95 kubernetes.io/service-account-token 3 64m
deck-token-j8pr2 kubernetes.io/service-account-token 3 64m
default-token-ltfch kubernetes.io/service-account-token 3 65m
expose-token-xl22f kubernetes.io/service-account-token 3 64m
hmac-token Opaque 1 64m
hook-token-xzkws kubernetes.io/service-account-token 3 64m
horologium-token-vc2tm kubernetes.io/service-account-token 3 64m
jenkins-docker-cfg Opaque 1 64m
jenkins-maven-settings Opaque 1 64m
jenkins-npm-token Opaque 1 64m
jenkins-release-gpg Opaque 4 64m
jenkins-ssh-config Opaque 1 64m
jenkins-x-chartmuseum Opaque 2 64m
jenkins-x-controllerbuild-token-zcq4n kubernetes.io/service-account-token 3 64m
jenkins-x-controllercommitstatus-token-fr8mn kubernetes.io/service-account-token 3 64m
jenkins-x-controllerrole-token-fnqpt kubernetes.io/service-account-token 3 64m
jenkins-x-controllerteam-token-4swph kubernetes.io/service-account-token 3 64m
jenkins-x-controllerworkflow-token-lk628 kubernetes.io/service-account-token 3 64m
jenkins-x-docker-registry-secret Opaque 1 64m
jenkins-x-gcactivities-token-f2c4l kubernetes.io/service-account-token 3 64m
jenkins-x-gcpods-token-xzwhd kubernetes.io/service-account-token 3 64m
jenkins-x-gcpreviews-token-2j4vr kubernetes.io/service-account-token 3 64m
jenkins-x-heapster-token-nbpx9 kubernetes.io/service-account-token 3 64m
jenkins-x-mongodb Opaque 1 64m
jx-basic-auth Opaque 1 64m
jx-install-config Opaque 2 64m
jx-pipeline-git-github-github Opaque 2 56m
knative-git-user-pass kubernetes.io/basic-auth 2 64m
nexus Opaque 1 64m
oauth-token Opaque 1 64m
pipeline-token-rmpfb kubernetes.io/service-account-token 3 64m
pipelinerunner-token-wd478 kubernetes.io/service-account-token 3 64m
plank-token-m9m2k kubernetes.io/service-account-token 3 64m
prow-build-token-r258m kubernetes.io/service-account-token 3 64m
sinker-token-z7ss9 kubernetes.io/service-account-token 3 64m
tekton-bot-token-lw4xs kubernetes.io/service-account-token 3 64m
tekton-pipelines-token-xq2h7 kubernetes.io/service-account-token 3 64m
tide-token-58fjm kubernetes.io/service-account-token 3 64m
webhook-certs Opaque 3 64m
INFO[0007]

[ccojocar]

@stevef1uk How did you expose the prow hook with minikube? This endpoint needs to be reachable from GitHub.

[stevef1uk]

I tried to use a sub domain DNS record to route through to my mac from the internet and then used kubectl port-forward on the mac. However, I had to guess that I needed to route to the 'hook' service. I manually edited the GitHub Webhooks, but wasn't able to get it to redeliver successfully. I know the route through to my mac worked (having opened a port on my internet router for this) as I stood up a temporary web page to check that part. I am guessing that the port forwarding to prow on minikube didn't work. Any clue as to which service / pod/port I should be targeting?

Also, I am surprised that a call from GitHub is required to build the code initially?

[jenkins-x-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle stale

[jenkins-x-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle rotten

[jenkins-x-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://jenkins-x.io/community.
/close

[jenkins-x-bot]

@jenkins-x-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://jenkins-x.io/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[ginosubscriptions]

I'm actually having the exact same problem on the latest JX3 version with Minikube. jx create quickstart completes successfully, my webhooks are reachable from outside the Minikube cluster (i.e.: GitHub can actually complete pull requests => it does it for the jx3-minikube repository). Still, once the entire creation process has run, no application is available (from jx get applications).
Not clear what's happening. The only thing I can say is that I tested it with the GCP/GKE setup and one of the big differences I see between the Minikube install of JX3 and the Google Kubernetes Engine one is that there are no Nexus or Helm repositories created (also no ingresses corresponding with these). Maybe that is normal, but I would consider those key components for deployments. I also see no "build job" logs with jx get build logs for the quickstart.

[ginosubscriptions]

Given my previous comment on "it's happening to me too", would it be possible to reopen this issue, or should I clone and create a new one?