Failed to create GCP PubSub EventSource

[jacobhagstedt]

Describe the bug
I've installed Argo-Events from fresh using kustomize set to tag v1.3.0 by using this kustomization file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
    - namespace.yaml
    - github.com/argoproj/argo-events/manifests/cluster-install?ref=v1.3.0
    - operate-workflow-role.yaml
    - workflow-role.yaml
namespace: argo-events

namespace.yaml:

apiVersion: v1
kind: Namespace
metadata:
  name: argo-events

I am running Argo server version v3.0.2

When I try to create a GCP PubSub EventSource using the example provided here (but changing the topic and project to fit me) I get the following error:

{"level":"info","ts":1619788468.9547193,"logger":"argo-events.eventsource","caller":"cmd/main.go:63","msg":"starting eventsource server","eventSourceName":"gcp-pubsub","version":"vv1.3.0+7591146.dirty"}
{"level":"info","ts":1619788468.9548414,"logger":"argo-events.eventsource","caller":"metrics/metrics.go:172","msg":"starting metrics server","eventSourceName":"gcp-pubsub"}
{"level":"fatal","ts":1619788468.9613414,"logger":"argo-events.eventsource","caller":"leaderelection/leaderelection.go:111","msg":"failed to new a node","eventSourceName":"gcp-pubsub","error":"open /tmp/graft.log: no such file or directory","stacktrace":"github.com/argoproj/argo-events/common/leaderelection.(*natsEventBusElector).RunOrDie\n\t/home/runner/work/argo-events/argo-events/common/leaderelection/leaderelection.go:111\ngithub.com/argoproj/argo-events/eventsources.(*EventSourceAdaptor).Start\n\t/home/runner/work/argo-events/argo-events/eventsources/eventing.go:293\nmain.main\n\t/home/runner/work/argo-events/argo-events/eventsources/cmd/main.go:65\nruntime.main\n\t/opt/hostedtoolcache/go/1.14.15/x64/src/runtime/proc.go:203"}

This results in the pod for the eventsource not getting started and gets stuck in a crash loop due to this "error":"open /tmp/graft.log: no such file or directory" error.

To Reproduce
Steps to reproduce the behavior:

1. kubectl apply -k kustomization.yaml on the above file (roles provided by these)
2. Deploy eventbus from here
3. When things are up and running, install the eventsource
4. Now the pod will get started by the controller and crash, see error above

Expected behavior
I expect the event source to be created successfully by the controller and successfully talking to the eventbus and connecting to the pubsub subscription!

Environment:

• Kubernetes: v1.17.17
• Argo: v3.0.2
• Argo Events: v1.3.0

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[whynowy]

If this issue happens in your cluster, you can easily reproduce it with a simple Calendar EventSource like below, and you will not find an emptyDir volume mounted to /tmp by checking the Event Source Pod yaml, which is supposed to be there (event-source-controller is supposed to add that automatically).

apiVersion: argoproj.io/v1alpha1
kind: EventSource
metadata:
  name: calendar
spec:
  calendar:
    example1:
      interval: 30s

However, I wasn't able to reproduce it by using the same approach to install v1.3.0 to a k8s cluster with the same version as yours....

[jacobhagstedt]

Ah, thanks for the hint, I'll try that one and see if it works!

If it fails to mount an emptyDir, could it be that I messed up some service account? If so, any tips on where I can look?

[whynowy]

Ah, thanks for the hint, I'll try that one and see if it works!

If it fails to mount an emptyDir, could it be that I messed up some service account? If so, any tips on where I can look?

Just found the bug, I'll put a fix quickly and get it released in v1.3.1.

You can workaround the issue by using an EventBus with token auth strategy:

apiVersion: argoproj.io/v1alpha1
kind: EventBus
metadata:
  name: default
spec:
  nats:
    native:
      auth: token