Argo workflow controller unable to pull images from private registry

[SiddharthMalhotra]

Facing the following issue on pulling image from private registry running on the host/server. Our cluster uses the microk8s distribution. The image built is a custom argoexec image with some certificates for authenticating minio.

Events:
Type Reason Age From Message

---

Normal Scheduled 42m default-scheduler Successfully assigned argo/subtraction-p75hn-2254700841 to dgx-station
Normal Pulling 41m (x4 over 42m) kubelet Pulling image "private-repository-k8s.local:31320/image-argoexec-v2.5.2-with-certs:2021-03-11T09-59-32"
Warning Failed 41m (x4 over 42m) kubelet Failed to pull image "private-repository-k8s.local:31320/image-argoexec-v2.5.2-with-certs:2021-03-11T09-59-32": rpc error: code = Unknown desc = failed to resolve image "private-repository-k8s.local:31320/image-argoexec-v2.5.2-with-certs:2021-03-11T09-59-32": no available registry endpoint: failed to do request: Head "https://private-repository-k8s.local:31320/v2/image-argoexec-v2.5.2-with-certs/manifests/2021-03-11T09-59-32": Forbidden
Warning Failed 41m (x4 over 42m) kubelet Error: ErrImagePull
Warning Failed 40m (x6 over 42m) kubelet Error: ImagePullBackOff
Normal BackOff 2m29s (x178 over 42m) kubelet Back-off pulling image "private-repository-k8s.local:31320/image-argoexec-v2.5.2-with-certs:2021-03-11T09-59-32"

In order to push image to the private registry we set the proxy values in the following locations:

/etc/systemd/system/docker.service.d/proxy.conf
[Service]
Environment="NO_PROXY=localhost,5.5.0.70"

Microk8s to pull docker images
/var/snap/microk8s/current/args/containerd-env
NO_PROXY=localhost,5.5.0.70

How can we enable the pod to pull images from private registry? Do we have to set proxy in the pod?