Cannot make start with devcontainer

[panicboat]

When I tried to start the development environment using vscode, I got the following error.
I started using devcontainer with vscode and executed make start API=true after `make.
How do I get it to start?

vscode ➜ ~/go/src/github.com/argoproj/argo-workflows (main) $ make start API=true
GIT_COMMIT=a7a72e7ca3c2bbf127b65fdcf1b5b85fbd2aad19 GIT_BRANCH=main GIT_TAG=untagged GIT_TREE_STATE=clean RELEASE_TAG=false DEV_BRANCH=false VERSION=latest
KUBECTX=k3d-k3s-default K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/arm64
RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false  STATIC_FILES=false ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true
starting STATIC_FILES=false (DEV_BRANCH=false, GIT_BRANCH=main), AUTH_MODE=hybrid, RUN_MODE=local, MANAGED_NAMESPACE=argo 
⚠️  not starting UI. If you want to test the UI, run 'make start UI=true' to start it
# Check dex, minio, postgres and mysql are in hosts file
grep '127.0.0.1.*azurite' /etc/hosts
127.0.0.1       azurite
grep '127.0.0.1.*minio' /etc/hosts
127.0.0.1       minio
grep '127.0.0.1.*postgres' /etc/hosts
127.0.0.1       postgres
grep '127.0.0.1.*mysql' /etc/hosts
127.0.0.1       mysql
env DEFAULT_REQUEUE_TIME=1s  ARGO_SECURE=false  ALWAYS_OFFLOAD_NODE_STATUS=false ARGO_LOGLEVEL=debug UPPERIO_DB_DEBUG=0 ARGO_AUTH_MODE=hybrid ARGO_NAMESPACED=true ARGO_NAMESPACE=argo  ARGO_MANAGED_NAMESPACE=argo  ARGO_EXECUTOR_PLUGINS=true ARGO_POD_STATUS_CAPTURE_FINALIZER=true ARGO_UI_SECURE=false  PROFILE=minimal kit controller server
[go-deps] (running)  job running
[go-deps] (running)  starting process
starting Kit server on http://localhost:3000
[go-deps] (succeeded)  queuing "build-argo"
[go-deps] (succeeded)  queuing "install"
[go-deps] (succeeded)  queuing "build-controller"
[build-controller] (waiting)  waiting for mutex
[build-controller] (waiting)  acquired mutex
[build-controller] (running)  job running
[build-argo] (waiting)  waiting for mutex
[install] (waiting)  waiting for mutex
[install] (waiting)  acquired mutex
[install] (running)  job running
[install] (running)  starting process
[build-controller] (running)  starting process
[build-controller] (running)  make[1]: Entering directory '/home/vscode/go/src/github.com/argoproj/argo-workflows'
[install] (running)  make[1]: Entering directory '/home/vscode/go/src/github.com/argoproj/argo-workflows'
[build-controller] (running)  GIT_COMMIT=a7a72e7ca3c2bbf127b65fdcf1b5b85fbd2aad19 GIT_BRANCH=main GIT_TAG=untagged GIT_TREE_STATE=clean RELEASE_TAG=false DEV_BRANCH=false VERSION=latest
[install] (running)  GIT_COMMIT=a7a72e7ca3c2bbf127b65fdcf1b5b85fbd2aad19 GIT_BRANCH=main GIT_TAG=untagged GIT_TREE_STATE=clean RELEASE_TAG=false DEV_BRANCH=false VERSION=latest
[build-controller] (running)  KUBECTX=k3d-k3s-default K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/arm64
[build-controller] (running)  RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false  STATIC_FILES=true ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true
[install] (running)  KUBECTX=k3d-k3s-default K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/arm64
[install] (running)  RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false  STATIC_FILES=true ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true
[install] (running)  kubectl get ns argo  || kubectl create ns argo 
[install] (running)  NAME   STATUS   AGE
[install] (running)  argo   Active   52d
[install] (running)  kubectl config set-context --current --namespace=argo 
[install] (running)  Context "k3d-k3s-default" modified.
[install] (running)  installing PROFILE=minimal
[install] (running)  kubectl kustomize --load-restrictor=LoadRestrictionsNone test/e2e/manifests/minimal \
[install] (running)     | sed 's|quay.io/argoproj/|quay.io/argoproj/|' \
[install] (running)     | sed 's/namespace: argo/namespace: argo /' \
[install] (running)     | KUBECTL_APPLYSET=true kubectl -n argo  apply --applyset=configmaps/install --server-side --prune -f -
[install] (running)  customresourcedefinition.apiextensions.k8s.io/clusterworkflowtemplates.argoproj.io serverside-applied
[install] (running)  customresourcedefinition.apiextensions.k8s.io/cronworkflows.argoproj.io serverside-applied
[install] (running)  customresourcedefinition.apiextensions.k8s.io/eventbus.argoproj.io serverside-applied
[install] (running)  customresourcedefinition.apiextensions.k8s.io/eventsources.argoproj.io serverside-applied
[install] (running)  customresourcedefinition.apiextensions.k8s.io/sensors.argoproj.io serverside-applied
[build-controller] (running)  go install k8s.io/kube-openapi/cmd/openapi-gen@v0.0.0-20220124234850-424119656bbf
[install] (running)  customresourcedefinition.apiextensions.k8s.io/workflowartifactgctasks.argoproj.io serverside-applied
[install] (running)  customresourcedefinition.apiextensions.k8s.io/workfloweventbindings.argoproj.io serverside-applied
[install] (running)  customresourcedefinition.apiextensions.k8s.io/workflows.argoproj.io serverside-applied
[install] (running)  customresourcedefinition.apiextensions.k8s.io/workflowtaskresults.argoproj.io serverside-applied
[install] (running)  customresourcedefinition.apiextensions.k8s.io/workflowtasksets.argoproj.io serverside-applied
[install] (running)  customresourcedefinition.apiextensions.k8s.io/workflowtemplates.argoproj.io serverside-applied
[install] (running)  serviceaccount/argo serverside-applied
[install] (running)  serviceaccount/argo-server serverside-applied
[install] (running)  serviceaccount/get-cm serverside-applied
[install] (running)  serviceaccount/github.com serverside-applied
[install] (running)  role.rbac.authorization.k8s.io/agent serverside-applied
[install] (running)  role.rbac.authorization.k8s.io/argo-role serverside-applied
[install] (running)  role.rbac.authorization.k8s.io/artifactgc serverside-applied
[install] (running)  role.rbac.authorization.k8s.io/executor serverside-applied
[install] (running)  role.rbac.authorization.k8s.io/pod-manager serverside-applied
[install] (running)  role.rbac.authorization.k8s.io/submit-workflow-template serverside-applied
[install] (running)  role.rbac.authorization.k8s.io/workflow-manager serverside-applied
[install] (running)  clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-admin serverside-applied
[install] (running)  clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-edit serverside-applied
[install] (running)  clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-view serverside-applied
[install] (running)  clusterrole.rbac.authorization.k8s.io/argo-cluster-role serverside-applied
[install] (running)  clusterrole.rbac.authorization.k8s.io/argo-clusterworkflowtemplate-role serverside-applied
[install] (running)  clusterrole.rbac.authorization.k8s.io/argo-server-cluster-role serverside-applied
[install] (running)  clusterrole.rbac.authorization.k8s.io/argo-server-clusterworkflowtemplate-role serverside-applied
[install] (running)  clusterrole.rbac.authorization.k8s.io/get-cm serverside-applied
[install] (running)  rolebinding.rbac.authorization.k8s.io/agent-default serverside-applied
[install] (running)  rolebinding.rbac.authorization.k8s.io/argo-binding serverside-applied
[install] (running)  rolebinding.rbac.authorization.k8s.io/artifactgc-default serverside-applied
[install] (running)  rolebinding.rbac.authorization.k8s.io/executor-default serverside-applied
[install] (running)  rolebinding.rbac.authorization.k8s.io/executor-get-cm serverside-applied
[install] (running)  rolebinding.rbac.authorization.k8s.io/github.com serverside-applied
[install] (running)  rolebinding.rbac.authorization.k8s.io/pod-manager-default serverside-applied
[install] (running)  rolebinding.rbac.authorization.k8s.io/workflow-manager-default serverside-applied
[install] (running)  clusterrolebinding.rbac.authorization.k8s.io/argo-binding serverside-applied
[install] (running)  clusterrolebinding.rbac.authorization.k8s.io/argo-clusterworkflowtemplate-role-binding serverside-applied
[install] (running)  clusterrolebinding.rbac.authorization.k8s.io/argo-server-binding serverside-applied
[install] (running)  clusterrolebinding.rbac.authorization.k8s.io/argo-server-clusterworkflowtemplate-role-binding serverside-applied
[install] (running)  clusterrolebinding.rbac.authorization.k8s.io/get-cm-get-cm serverside-applied
[install] (running)  configmap/artifact-repositories serverside-applied
[install] (running)  configmap/workflow-controller-configmap serverside-applied
[install] (running)  secret/argo-server-sso serverside-applied
[install] (running)  secret/argo-server.service-account-token serverside-applied
[install] (running)  secret/argo-workflows-agent-ca-certificates serverside-applied
[install] (running)  secret/argo-workflows-webhook-clients serverside-applied
[install] (running)  secret/argo.service-account-token serverside-applied
[install] (running)  secret/default.service-account-token serverside-applied
[install] (running)  secret/get-cm.service-account-token serverside-applied
[build-controller] (running)  go: downloading k8s.io/kube-openapi v0.0.0-20220124234850-424119656bbf
[install] (running)  secret/github.com.service-account-token serverside-applied
[install] (running)  secret/my-httpbin-cred serverside-applied
[install] (running)  secret/my-minio-cred serverside-applied
[install] (running)  service/argo-server serverside-applied
[install] (running)  service/httpbin serverside-applied
[install] (running)  service/minio serverside-applied
[install] (running)  priorityclass.scheduling.k8s.io/workflow-controller serverside-applied
[install] (running)  Error from server (Invalid): Deployment.apps "argo-server" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"argo-server"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
[install] (running)  Error from server (Invalid): Deployment.apps "httpbin" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"httpbin"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
[install] (running)  Error from server (Invalid): Deployment.apps "minio" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"minio"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
[install] (running)  Error from server (Invalid): Deployment.apps "workflow-controller" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"workflow-controller"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
[install] (running)  make[1]: Leaving directory '/home/vscode/go/src/github.com/argoproj/argo-workflows'
[install] (running)  make[1]: *** [Makefile:483: install] Error 1
[install] (failed)  exit status 2
exiting because a job failed
waiting for all tasks to complete
[build-controller] (running)  make[1]: *** [Makefile:340: /home/vscode/go/bin/openapi-gen] Terminated
[build-controller] (succeeded)  cancelled
[build-argo] (waiting)  acquired mutex
[build-argo] (running)  job running
[build-argo] (running)  starting process
[build-argo] (succeeded)  cancelled
[build-argo] (succeeded) cancelled
[controller] (pending) 
[install] (failed) exit status 2
[go-deps] (succeeded) 
[build-controller] (succeeded) cancelled
[port-forward] (pending) 
[server] (pending) 
failed tasks: [install]
make: *** [Makefile:555: start] Error 1

[MasonM]

This might have been a side-effect of the switch to applysets in #14048. This should fix it: kubectl delete --all deploy

[MasonM]

@panicboat I think I figured that out. The error you posted is pointing to:

argo-workflows/Makefile

Line 559 in 8304dc7

ifneq ($(PLUGINS),true)

which expands to

argo-workflows/Makefile

Line 54 in 8304dc7

PLUGINS ?= $(shell [ $PROFILE = plugins ] && echo false || echo true)

but $PROFILE isn't being escaped properly, which means if you have the P variable set to something, it'll break. I included a fix for it in #14094

[panicboat]

@MasonM @tczhao Thank you for your confirmation and response!
I will now be able to contribute!

[panicboat]

Hi.
I tried updating the code and make start UI=true but that did not solve the problem.
(I have confirmed that the fixes in #13922 have been incorporated.)

In addition, the steps we performed are as follows.

1. make clean
2. kubectl delete --all deploy
3. make
4. make start UI=true

vscode ➜ ~/go/src/github.com/argoproj/argo-workflows (main) $ make start UI=true
GIT_COMMIT=514ea3741bd1ce151ce51eba8686ae8e7ef91105 GIT_BRANCH=main GIT_TAG=untagged GIT_TREE_STATE=dirty RELEASE_TAG=false DEV_BRANCH=false VERSION=latest
KUBECTX=k3d-k3s-default K3D=true DOCKER_PUSH=false TARGET_PLATFORM=linux/arm64
RUN_MODE=local PROFILE=minimal AUTH_MODE=hybrid SECURE=false  STATIC_FILES=false ALWAYS_OFFLOAD_NODE_STATUS=false UPPERIO_DB_DEBUG=0 LOG_LEVEL=debug NAMESPACED=true
starting STATIC_FILES=false (DEV_BRANCH=false, GIT_BRANCH=main), AUTH_MODE=hybrid, RUN_MODE=local, MANAGED_NAMESPACE=argo 
⚠️️  not starting API. If you want to test the API, use 'make start API=true' to start it
⚠️  not starting plugins. If you want to test plugins, run 'make start PROFILE=plugins' to start it
# Check dex, minio, postgres and mysql are in hosts file
grep '127.0.0.1.*azurite' /etc/hosts
127.0.0.1       azurite
grep '127.0.0.1.*minio' /etc/hosts
127.0.0.1       minio
grep '127.0.0.1.*postgres' /etc/hosts
127.0.0.1       postgres
grep '127.0.0.1.*mysql' /etc/hosts
127.0.0.1       mysql
env DEFAULT_REQUEUE_TIME=1s  ARGO_SECURE=false  ALWAYS_OFFLOAD_NODE_STATUS=false ARGO_LOGLEVEL=debug UPPERIO_DB_DEBUG=0 ARGO_AUTH_MODE=hybrid ARGO_NAMESPACED=true ARGO_NAMESPACE=argo  ARGO_MANAGED_NAMESPACE=argo  ARGO_EXECUTOR_PLUGINS=false ARGO_POD_STATUS_CAPTURE_FINALIZER=true ARGO_UI_SECURE=false  PROFILE=minimal kit controller server ui
no such file or directory
make: *** [Makefile:560: start] Error 1

There was no problem with make start or make start API=true.

[MasonM]

@panicboat Thanks for your persistence. I was able to reproduce it too, and I think it's a backwards-incompatible change with kit: kitproj/kit#81

Workaround: delete the following line from tasks.yaml:

argo-workflows/tasks.yaml

Line 79 in 514ea37

watch: ui/package.json ui/yarn.lock

[panicboat]

Thank you for your response, it started up fine.

[jswxstw]

#14105 should have fixed it.

[panicboat]

Thank you for your response, it started up fine.