Wrong service name for actor system!? #123

janpieper · 2023-01-19T14:25:26Z

Describe the bug

I don't know whether this is a bug or I am doing something wrong, but at least it feels/looks wrong to me 🤔

When I create an actor system named example I get a service named system-spawn-system-svc:

apiVersion: v1
kind: Service
metadata:
  name: system-spawn-system-svc
spec:
  clusterIP: None
  clusterIPs:
  - None
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: epmd
    port: 4369
    protocol: TCP
    targetPort: epmd
  selector:
    actor-system: spawn-system
  sessionAffinity: None
  type: ClusterIP

To Reproduce

apiVersion: eigr-spawn.io/v1
kind: ActorSystem
metadata:
  name: example
spec:
  # ...

Expected behavior

Service is named system-example-svc and uses actor-system: example as selector.

apiVersion: v1
kind: Service
metadata:
  name: system-example-svc
spec:
  clusterIP: None
  clusterIPs:
  - None
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - name: epmd
    port: 4369
    protocol: TCP
    targetPort: epmd
  selector:
    actor-system: example
  sessionAffinity: None
  type: ClusterIP

_{Btw.: It feels redundant to name a service "*-svc"}

Kubernetes:

Version v1.25.2
Provider: Bare metal 😉

Spawn Operator:

Version eigr/spawn-operator:0.5.0-rc.9

The text was updated successfully, but these errors were encountered:

sleipnir · 2023-01-19T14:48:15Z

Hi @janpieper thanks for opening this issue.

We have some points here and I will try to answer them one by one.
First of all this is not a bug as it does not affect functionality, so I will remove the bug label from the issue.

Regarding the suffix I agree that this is unnecessary, I even had a mental note to do it here. This was inherited from our old Massa proxy that ended up being left by the wayside. I will create an issue to specifically address this point.

Regarding the selector, I don't know if this would be relevant, but we can consider doing it. Do you have any particular reason to keep the selector exactly as you named it? I ask this because all resources created are managed by the Operator and not the end user, so I would like to know what impact this, if any, has on your operation?

janpieper · 2023-01-19T15:15:42Z

I understand these resources are created and managed by the operator 😉 That's not the problem (at least from my understanding).

Maybe I am still missing an overall understanding about all the components, so please do not take everything too serious I am asking here 😅 I am not looking into this fulltime yet 😅 I am still missing a lot of knowledge about the whole system.

FYI: I am using the term "Pod" in the following graphs, because I am still unsure about what those are in the context of spawn! (maybe actors?)

At the moment it feels like when creating two actor systems, only one service gets created.

graph LR
  ActorSystemOne[ActorSystem: one]
  ActorSystemTwo[ActorSystem: two]
  Service["Service: system-spawn-system-svc<br /><br />selector(actor-system: spawn-system)"]

  ActorSystemOne-->Service
  ActorSystemTwo-->Service
  Service-->PodOne[Pod]
  Service-->PodTwo[Pod]
  Service-->PodThree[Pod]

For my understanding so far, I would expect the operator to create two separate services with selectors that match the actor system 🤔

graph LR
  ActorSystemOne[ActorSystem: one]
  ActorSystemTwo[ActorSystem: two]
  ServiceOne["Service: system-one-svc<br /><br />select(actor-system: one)"]
  ServiceTwo["Service: system-two-svc<br /><br />selector(actor-system: two)"]

  ActorSystemOne-->ServiceOne
  ActorSystemTwo-->ServiceTwo
  ServiceOne-->PodOne[Pod]
  ServiceOne-->PodTwo[Pod]
  ServiceOne-->PodThree[Pod]
  ServiceTwo-->PodFour[Pod]
  ServiceTwo-->PodFive[Pod]
  ServiceTwo-->PodSix[Pod]

sleipnir · 2023-01-19T15:42:10Z

@janpieper Hello very nice diagrams.
I said resources are managed just as a hook to my question about whether there was any impact on its operation, don't take me too seriously either :D

Let me try to explain the concepts of system.

An ActorSystem works as a bucket for ActorHosts. In turn, an ActorHost is the client application + the proxy. An ActorHost can contain as many actors as the user defines and an ActorSystem can contain N ActorHosts.

This bucket, ActorSystem, in turn is closed in itself, that is, only ActorHosts that are part of the same ActorSystem can form a cluster.
If you define two ActorSystems then it is to be expected that you want to create two different clusters.

Having said all that, Pods for us are exactly what they are for Kuberntes and in our case a Pod created by our Operator will always contain a container with the user application and another with the proxy, that is, our sidecar. Unless the user uses the SDK for Elixir. In this case, the Pod will only contain the container with the user's application, since, in this case, the Elixir SDK itself will connect to the ActorSystem Cluster of which it is a part, without the need for a proxy.

Talking a little more about the CRD ActorSystem. This CRD will create a Headless service type exposing the epmd port so that proxies can form an Erlang Dist cluster. It will also create secrets and configmaps to configure the proxies according to the user's wishes.

After creating an ActorSystem you must create an ActorHost defining your application itself. At this moment another service will be created exposing the ports that you want to expose from your application.

Let me know if this explanation helped you or if you still have questions about the process.

sleipnir · 2023-01-19T15:57:08Z

I have tried to illustrate the concepts with the diagram below:

janpieper · 2023-01-19T16:05:13Z

Maybe I am a bit confused 😅 Your diagram looks like I would expect it to be, but at the moment, the operator only creates a single "Headless Svc" for both ActorSystems. So, they share a single service.

I updated your diagram to look like I currently see it on my Kubernetes:

janpieper · 2023-01-19T16:06:11Z

Is it allowed to have 1+ ActorSystem per namespace?

sleipnir · 2023-01-19T16:20:51Z

@janpieper This is not the correct behavior, please send me the output of the commands below:

kubectl -n your_namespace get system
kubectl -n your_namespace get svc
kubectl -n eigr-functions get all

As well as operator pod logs

sleipnir · 2023-01-19T16:21:32Z

Is it allowed to have 1+ ActorSystem per namespace?

You have to be able to create as many ActorSystems as you want in as many namespaces as you want too.

janpieper · 2023-01-19T16:36:37Z

One ActorSystem

kubectl -n your_namespace get system

$ kubectl get system -n yggdrasil
NAME        AGE
yggdrasil   3h35m

kubectl -n your_namespace get svc

$ kubectl get services -n yggdrasil
NAME                      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
postgresql                ClusterIP   10.108.210.11   <none>        5432/TCP   8h
postgresql-hl             ClusterIP   None            <none>        5432/TCP   8h
postgresql-metrics        ClusterIP   10.97.231.38    <none>        9187/TCP   8h
system-spawn-system-svc   ClusterIP   None            <none>        4369/TCP   3h35m

kubectl -n eigr-functions get all

$ kubectl get all -n eigr-functions
NAME                                  READY   STATUS    RESTARTS   AGE
pod/spawn-operator-7c559c5598-bdcjt   1/1     Running   0          2d8h

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/spawn-operator   1/1     1            1           2d22h

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/spawn-operator-576f9488c9   0         0         0       2d19h
replicaset.apps/spawn-operator-679b5dfbc9   0         0         0       2d22h
replicaset.apps/spawn-operator-67c845c576   0         0         0       2d8h
replicaset.apps/spawn-operator-7c559c5598   1         1         1       2d8h
replicaset.apps/spawn-operator-86ccd45df    0         0         0       2d19h
replicaset.apps/spawn-operator-9cdf7b5c5    0         0         0       2d8h

As well as operator pod logs

$ kubectl logs -n eigr-functions --since=1m spawn-operator-7c559c5598-bdcjt
2023-01-19 16:27:34.155 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:35.111 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:36.401 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:37.711 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13125.24> ]:[info]:GET /health
2023-01-19 16:27:37.711 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13125.24> ]:[info]:GET /health
2023-01-19 16:27:37.711 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13125.24> ]:[info]:Sent 200 in 80µs
2023-01-19 16:27:37.711 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13125.24> ]:[info]:Sent 200 in 222µs
2023-01-19 16:27:39.165 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:40.120 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:41.413 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:44.194 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:45.128 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:45.282 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6808.23> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:27:45.300 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13038.24> ]:[debug]:Reconciler fetch succeeded
2023-01-19 16:27:45.300 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13063.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Processing event
2023-01-19 16:27:45.300 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13063.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Applying status
2023-01-19 16:27:45.311 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13063.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Emitting Normal event
2023-01-19 16:27:45.321 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13038.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:27:46.442 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:47.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13157.24> ]:[info]:GET /health
2023-01-19 16:27:47.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13157.24> ]:[info]:GET /health
2023-01-19 16:27:47.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13157.24> ]:[info]:Sent 200 in 127µs
2023-01-19 16:27:47.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13157.24> ]:[info]:Sent 200 in 390µs
2023-01-19 16:27:49.225 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:50.158 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:51.444 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:54.257 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:55.189 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:56.451 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:27:57.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13203.24> ]:[info]:GET /health
2023-01-19 16:27:57.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13203.24> ]:[info]:GET /health
2023-01-19 16:27:57.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13203.24> ]:[info]:Sent 200 in 147µs
2023-01-19 16:27:57.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13203.24> ]:[info]:Sent 200 in 475µs
2023-01-19 16:27:59.286 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:00.220 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:01.403 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2254.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:28:01.403 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2245.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:28:01.423 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13208.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:28:01.429 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13192.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:28:01.482 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:04.316 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:05.251 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:06.485 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:07.711 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13227.24> ]:[info]:GET /health
2023-01-19 16:28:07.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13227.24> ]:[info]:GET /health
2023-01-19 16:28:07.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13227.24> ]:[info]:Sent 200 in 134µs
2023-01-19 16:28:07.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13227.24> ]:[info]:Sent 200 in 371µs
2023-01-19 16:28:09.327 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:10.283 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:11.515 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:14.359 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:15.312 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:15.322 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6808.23> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:28:15.347 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13254.24> ]:[debug]:Reconciler fetch succeeded
2023-01-19 16:28:15.347 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13284.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Processing event
2023-01-19 16:28:15.348 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13284.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Applying status
2023-01-19 16:28:15.354 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13284.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Emitting Normal event
2023-01-19 16:28:15.367 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13254.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:28:16.548 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:17.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13276.24> ]:[info]:GET /health
2023-01-19 16:28:17.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13276.24> ]:[info]:GET /health
2023-01-19 16:28:17.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13276.24> ]:[info]:Sent 200 in 126µs
2023-01-19 16:28:17.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13276.24> ]:[info]:Sent 200 in 325µs
2023-01-19 16:28:19.390 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:20.342 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:21.557 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:24.425 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:25.375 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:26.567 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:27.711 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13324.24> ]:[info]:GET /health
2023-01-19 16:28:27.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13324.24> ]:[info]:GET /health
2023-01-19 16:28:27.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13324.24> ]:[info]:Sent 200 in 106µs
2023-01-19 16:28:27.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13324.24> ]:[info]:Sent 200 in 307µs
2023-01-19 16:28:29.457 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:30.407 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:31.424 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2254.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:28:31.433 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2245.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:28:31.447 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13313.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:28:31.451 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13323.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:28:31.573 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:34.487 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:35.442 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:36.577 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:37.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13359.24> ]:[info]:GET /health
2023-01-19 16:28:37.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13359.24> ]:[info]:GET /health
2023-01-19 16:28:37.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13359.24> ]:[info]:Sent 200 in 123µs
2023-01-19 16:28:37.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13359.24> ]:[info]:Sent 200 in 355µs
2023-01-19 16:28:39.491 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:40.459 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:41.606 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:44.523 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:45.368 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6808.23> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:28:45.403 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13369.24> ]:[debug]:Reconciler fetch succeeded
2023-01-19 16:28:45.404 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13293.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Processing event
2023-01-19 16:28:45.405 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13293.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Applying status
2023-01-19 16:28:45.416 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13293.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Emitting Normal event
2023-01-19 16:28:45.423 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13369.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:28:45.468 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:46.634 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:47.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13426.24> ]:[info]:GET /health
2023-01-19 16:28:47.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13426.24> ]:[info]:GET /health
2023-01-19 16:28:47.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13426.24> ]:[info]:Sent 200 in 122µs
2023-01-19 16:28:47.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13426.24> ]:[info]:Sent 200 in 356µs
2023-01-19 16:28:49.554 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:50.479 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:51.665 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:54.587 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:55.510 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:56.690 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:28:57.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13443.24> ]:[info]:GET /health
2023-01-19 16:28:57.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13443.24> ]:[info]:GET /health
2023-01-19 16:28:57.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13443.24> ]:[info]:Sent 200 in 120µs
2023-01-19 16:28:57.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13443.24> ]:[info]:Sent 200 in 339µs
2023-01-19 16:28:59.613 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:00.517 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:01.447 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2254.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:29:01.451 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2245.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:29:01.458 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13432.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:29:01.463 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13444.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:29:01.721 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:04.643 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:05.544 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:06.733 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:07.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13482.24> ]:[info]:GET /health
2023-01-19 16:29:07.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13482.24> ]:[info]:GET /health
2023-01-19 16:29:07.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13482.24> ]:[info]:Sent 200 in 193µs
2023-01-19 16:29:07.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13482.24> ]:[info]:Sent 200 in 483µs
2023-01-19 16:29:09.654 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:10.552 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:11.760 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:14.686 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:15.423 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6808.23> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:29:15.435 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13503.24> ]:[debug]:Reconciler fetch succeeded
2023-01-19 16:29:15.436 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13394.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Processing event
2023-01-19 16:29:15.436 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13394.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Applying status
2023-01-19 16:29:15.441 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13394.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Emitting Normal event
2023-01-19 16:29:15.446 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13503.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:29:15.581 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:16.788 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:17.711 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13504.24> ]:[info]:GET /health
2023-01-19 16:29:17.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13504.24> ]:[info]:GET /health
2023-01-19 16:29:17.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13504.24> ]:[info]:Sent 200 in 150µs
2023-01-19 16:29:17.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13504.24> ]:[info]:Sent 200 in 432µs
2023-01-19 16:29:19.725 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:20.591 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:21.818 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:24.754 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:25.620 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:26.844 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:27.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13546.24> ]:[info]:GET /health
2023-01-19 16:29:27.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13546.24> ]:[info]:GET /health
2023-01-19 16:29:27.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13546.24> ]:[info]:Sent 200 in 134µs
2023-01-19 16:29:27.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13546.24> ]:[info]:Sent 200 in 360µs
2023-01-19 16:29:29.781 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:30.646 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:29:31.458 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2254.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:29:31.463 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2245.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:29:31.466 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13564.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:29:31.471 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.13548.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:29:31.872 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch

janpieper · 2023-01-19T16:39:21Z

Two ActorSystems

kubectl -n your_namespace get system

$ kubectl get system -n yggdrasil
NAME        AGE
hugin       3m7s
yggdrasil   3h46m

kubectl -n your_namespace get svc

$ kubectl get services -n yggdrasil
NAME                      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
postgresql                ClusterIP   10.108.210.11   <none>        5432/TCP   9h
postgresql-hl             ClusterIP   None            <none>        5432/TCP   9h
postgresql-metrics        ClusterIP   10.97.231.38    <none>        9187/TCP   9h
system-spawn-system-svc   ClusterIP   None            <none>        4369/TCP   3h47m

kubectl -n eigr-functions get all

$ kubectl get all -n eigr-functions
NAME                                  READY   STATUS    RESTARTS   AGE
NAME                                  READY   STATUS    RESTARTS   AGE
pod/spawn-operator-7c559c5598-bdcjt   1/1     Running   0          2d8h

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/spawn-operator   1/1     1            1           2d22h

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/spawn-operator-576f9488c9   0         0         0       2d19h
replicaset.apps/spawn-operator-679b5dfbc9   0         0         0       2d22h
replicaset.apps/spawn-operator-67c845c576   0         0         0       2d8h
replicaset.apps/spawn-operator-7c559c5598   1         1         1       2d8h
replicaset.apps/spawn-operator-86ccd45df    0         0         0       2d19h
replicaset.apps/spawn-operator-9cdf7b5c5    0         0         0       2d8h

As well as operator pod logs

$ kubectl logs -n eigr-functions --since=1m spawn-operator-7c559c5598-bdcjt
2023-01-19 16:37:54.294 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:37:56.396 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:37:57.039 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:37:57.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15682.24> ]:[info]:GET /health
2023-01-19 16:37:57.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15682.24> ]:[info]:GET /health
2023-01-19 16:37:57.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15682.24> ]:[info]:Sent 200 in 120µs
2023-01-19 16:37:57.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15682.24> ]:[info]:Sent 200 in 332µs
2023-01-19 16:37:59.320 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:01.430 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:01.745 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2245.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:38:01.751 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2254.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:38:01.772 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15735.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:38:01.776 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15737.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:38:02.051 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:04.350 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:06.447 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:07.056 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:07.711 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15743.24> ]:[info]:GET /health
2023-01-19 16:38:07.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15743.24> ]:[info]:GET /health
2023-01-19 16:38:07.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15743.24> ]:[info]:Sent 200 in 105µs
2023-01-19 16:38:07.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15743.24> ]:[info]:Sent 200 in 277µs
2023-01-19 16:38:09.360 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:11.476 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:12.063 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:14.388 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:16.504 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:17.078 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:17.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15766.24> ]:[info]:GET /health
2023-01-19 16:38:17.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15766.24> ]:[info]:GET /health
2023-01-19 16:38:17.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15766.24> ]:[info]:Sent 200 in 126µs
2023-01-19 16:38:17.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15766.24> ]:[info]:Sent 200 in 387µs
2023-01-19 16:38:19.420 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:20.962 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14718.24> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:38:20.970 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15767.24> ]:[debug]:Reconciler fetch succeeded
2023-01-19 16:38:20.970 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15767.24> ]:[debug]:Reconciler fetch succeeded
2023-01-19 16:38:20.970 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15776.24> ]:[info]:{"yggdrasil/hugin", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Processing event
2023-01-19 16:38:20.970 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15791.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Processing event
2023-01-19 16:38:20.970 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15791.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Applying status
2023-01-19 16:38:20.970 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15776.24> ]:[info]:{"yggdrasil/hugin", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Applying status
2023-01-19 16:38:20.978 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15791.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Emitting Normal event
2023-01-19 16:38:20.978 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15776.24> ]:[info]:{"yggdrasil/hugin", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Emitting Normal event
2023-01-19 16:38:20.989 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15767.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:38:21.517 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:22.109 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:24.446 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:26.526 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:27.113 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:27.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15801.24> ]:[info]:GET /health
2023-01-19 16:38:27.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15801.24> ]:[info]:GET /health
2023-01-19 16:38:27.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15801.24> ]:[info]:Sent 200 in 138µs
2023-01-19 16:38:27.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15801.24> ]:[info]:Sent 200 in 381µs
2023-01-19 16:38:29.477 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:31.558 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:31.773 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2245.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:38:31.778 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2254.0> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:38:31.806 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15812.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:38:31.808 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15746.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:38:32.140 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:34.507 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:36.567 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:37.152 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:37.711 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15873.24> ]:[info]:GET /health
2023-01-19 16:38:37.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15873.24> ]:[info]:GET /health
2023-01-19 16:38:37.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15873.24> ]:[info]:Sent 200 in 117µs
2023-01-19 16:38:37.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15873.24> ]:[info]:Sent 200 in 336µs
2023-01-19 16:38:39.512 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:41.600 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:42.178 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:44.539 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:46.626 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:47.212 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:47.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15885.24> ]:[info]:GET /health
2023-01-19 16:38:47.712 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15885.24> ]:[info]:GET /health
2023-01-19 16:38:47.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15885.24> ]:[info]:Sent 200 in 182µs
2023-01-19 16:38:47.713 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15885.24> ]:[info]:Sent 200 in 455µs
2023-01-19 16:38:49.571 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2265.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:50.990 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14718.24> ]:[debug]:AsyncStreamRunner - Starting 
2023-01-19 16:38:50.999 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15904.24> ]:[debug]:Reconciler fetch succeeded
2023-01-19 16:38:50.999 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15904.24> ]:[debug]:Reconciler fetch succeeded
2023-01-19 16:38:50.999 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15892.24> ]:[info]:{"yggdrasil/hugin", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Processing event
2023-01-19 16:38:50.999 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15892.24> ]:[info]:{"yggdrasil/hugin", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Applying status
2023-01-19 16:38:50.999 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15907.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Processing event
2023-01-19 16:38:50.999 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15907.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Applying status
2023-01-19 16:38:51.004 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15892.24> ]:[info]:{"yggdrasil/hugin", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Emitting Normal event
2023-01-19 16:38:51.008 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15907.24> ]:[info]:{"yggdrasil/yggdrasil", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:reconcile"} - Emitting Normal event
2023-01-19 16:38:51.012 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.15904.24> ]:[debug]:AsyncStreamRunner - Stream terminated
2023-01-19 16:38:51.658 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.14691.24> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch
2023-01-19 16:38:52.243 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.2251.0> ]:[debug]:K8s.Client.Runner.Watch.Stream - Watch request timed out - resuming the watch

janpieper · 2023-01-19T16:41:25Z

I've also seen errors in the operator log:

2023-01-19 16:33:50.225 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[info]:{"yggdrasil/hugin", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:add"} - Processing event
2023-01-19 16:33:50.256 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[error]:Task #PID<0.6809.23> started from #PID<0.6808.23> terminating
** (MatchError) no match of right hand side value: {:error, %K8s.Client.APIError{message: "secrets \"postgresql-creds\" not found", reason: "NotFound"}}
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/k8s/secret/actor_system_secret.ex:44: SpawnOperator.K8s.Secret.ActorSystemSecret.get_storage_options/3
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/k8s/secret/actor_system_secret.ex:25: SpawnOperator.K8s.Secret.ActorSystemSecret.gen_secret/1
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/handler/actor_system_handler.ex:41: SpawnOperator.Handler.ActorSystemHandler.call/2
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/controller/actor_system_controller.ex:1: SpawnOperator.Controller.ActorSystemController.pluggable_builder_call/2
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/operator.ex:1: SpawnOperator.Operator.pluggable_builder_call/2
    (bonny 1.0.0-rc.3) lib/bonny/operator.ex:149: Bonny.Operator.run/4
    (elixir 1.14.0) lib/stream.ex:612: anonymous fn/4 in Stream.map/2
    (elixir 1.14.0) lib/stream.ex:1651: Stream.do_element_resource/6
Function: &Bonny.Server.AsyncStreamRunner.run/2
    Args: [#Stream<[enum: #Function<52.124013645/2 in Stream.resource/3>, funs: [#Function<48.124013645/1 in Stream.map/2>, #Function<48.124013645/1 in Stream.map/2>]]>, nil]

The postgresql-creds secret exists in the yggdrasil namespace.

sleipnir · 2023-01-19T17:19:12Z

@janpieper Thanks for the info, I'll look into it

sleipnir · 2023-01-19T18:20:47Z

I've also seen errors in the operator log:

2023-01-19 16:33:50.225 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[info]:{"yggdrasil/hugin", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:add"} - Processing event
2023-01-19 16:33:50.256 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[error]:Task #PID<0.6809.23> started from #PID<0.6808.23> terminating
** (MatchError) no match of right hand side value: {:error, %K8s.Client.APIError{message: "secrets \"postgresql-creds\" not found", reason: "NotFound"}}
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/k8s/secret/actor_system_secret.ex:44: SpawnOperator.K8s.Secret.ActorSystemSecret.get_storage_options/3
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/k8s/secret/actor_system_secret.ex:25: SpawnOperator.K8s.Secret.ActorSystemSecret.gen_secret/1
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/handler/actor_system_handler.ex:41: SpawnOperator.Handler.ActorSystemHandler.call/2
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/controller/actor_system_controller.ex:1: SpawnOperator.Controller.ActorSystemController.pluggable_builder_call/2
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/operator.ex:1: SpawnOperator.Operator.pluggable_builder_call/2
    (bonny 1.0.0-rc.3) lib/bonny/operator.ex:149: Bonny.Operator.run/4
    (elixir 1.14.0) lib/stream.ex:612: anonymous fn/4 in Stream.map/2
    (elixir 1.14.0) lib/stream.ex:1651: Stream.do_element_resource/6
Function: &Bonny.Server.AsyncStreamRunner.run/2
    Args: [#Stream<[enum: #Function<52.124013645/2 in Stream.resource/3>, funs: [#Function<48.124013645/1 in Stream.map/2>, #Function<48.124013645/1 in Stream.map/2>]]>, nil]

The postgresql-creds secret exists in the yggdrasil namespace.

This is because we recently changed the Operator's type from Cluster to Namespaced. That way Operator will only be able to list resources in its own namespace. To avoid this, you can create the secret of the access data to the database in the same namespace as the Operator, in this case eigr-functions

janpieper · 2023-01-19T18:21:25Z

Okay, it seems like the kubernetes manifests shown in the examples are out of date 🤔

Have a look into fd03504. It seems like the following

apiVersion: spawn-eigr.io/v1
kind: ActorHost
metadata:
  name: example
  system: yggdrasil
# ...

needs to be changed into:

apiVersion: spawn-eigr.io/v1
kind: ActorHost
metadata:
  name: example
  labels:
    spawn-eigr.io.actor-system: yggdrasil
# ...

Without the spawn-eigr.io.actor-system label, the system name defaults to spawn-system.

Edit: Corrected commit hash.

sleipnir · 2023-01-19T19:06:55Z

@janpieper please try new rc.10 release https://github.com/eigr/spawn/releases/tag/v0.5.0-rc.10

janpieper · 2023-01-19T19:30:47Z

What about the docker image? 😉

sleipnir · 2023-01-19T19:33:07Z

What about the docker image? wink

Done

janpieper · 2023-01-19T19:47:23Z

Great 👍 Don't know whether I have time to have a look into this today. Will have a look tomorrow.

Thanks for your fast help 🙂

sleipnir · 2023-01-19T20:11:35Z

No problem @janpieper, in your time. It's a pleasure to be able to help.

janpieper · 2023-01-19T20:23:50Z

Where does the operator look for the credentialsSecretRef defined in the ActorSystem? 🤔 I am still getting the error that the operator does not find the secret although it exists in the yggrasil namespace.

2023-01-19 20:19:39.704 [spawn_operator@spawn-operator-5658954c5b-j9wqg]:[pid=<0.2775.0> ]:[error]:Task #PID<0.2775.0> started from #PID<0.2306.0> terminating
** (MatchError) no match of right hand side value: {:error, %K8s.Client.APIError{message: "secrets \"postgresql-creds\" not found", reason: "NotFound"}}
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/k8s/secret/actor_system_secret.ex:44: SpawnOperator.K8s.Secret.ActorSystemSecret.get_storage_options/3
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/k8s/secret/actor_system_secret.ex:25: SpawnOperator.K8s.Secret.ActorSystemSecret.gen_secret/1
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/handler/actor_system_handler.ex:42: SpawnOperator.Handler.ActorSystemHandler.call/2
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/controller/actor_system_controller.ex:1: SpawnOperator.Controller.ActorSystemController.pluggable_builder_call/2
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/operator.ex:1: SpawnOperator.Operator.pluggable_builder_call/2
    (bonny 1.0.0) lib/bonny/operator.ex:149: Bonny.Operator.run/4
    (elixir 1.14.0) lib/stream.ex:612: anonymous fn/4 in Stream.map/2
    (elixir 1.14.0) lib/stream.ex:1651: Stream.do_element_resource/6
Function: &Bonny.Server.AsyncStreamRunner.run/2
    Args: [#Stream<[enum: #Function<52.124013645/2 in Stream.resource/3>, funs: [#Function<48.124013645/1 in Stream.map/2>, #Function<48.124013645/1 in Stream.map/2>]]>, nil]

And what happened to the headless service? It is gone 😅 🤔

sleipnir · 2023-01-19T20:40:36Z

I've also seen errors in the operator log:

2023-01-19 16:33:50.225 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[info]:{"yggdrasil/hugin", "spawn-eigr.io/v1", "Kind=ActorSystem, Action=:add"} - Processing event
2023-01-19 16:33:50.256 [spawn_operator@spawn-operator-7c559c5598-bdcjt]:[pid=<0.6809.23> ]:[error]:Task #PID<0.6809.23> started from #PID<0.6808.23> terminating
** (MatchError) no match of right hand side value: {:error, %K8s.Client.APIError{message: "secrets \"postgresql-creds\" not found", reason: "NotFound"}}
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/k8s/secret/actor_system_secret.ex:44: SpawnOperator.K8s.Secret.ActorSystemSecret.get_storage_options/3
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/k8s/secret/actor_system_secret.ex:25: SpawnOperator.K8s.Secret.ActorSystemSecret.gen_secret/1
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/handler/actor_system_handler.ex:41: SpawnOperator.Handler.ActorSystemHandler.call/2
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/controller/actor_system_controller.ex:1: SpawnOperator.Controller.ActorSystemController.pluggable_builder_call/2
    (spawn_operator 0.0.0-local.dev) lib/spawn_operator/operator.ex:1: SpawnOperator.Operator.pluggable_builder_call/2
    (bonny 1.0.0-rc.3) lib/bonny/operator.ex:149: Bonny.Operator.run/4
    (elixir 1.14.0) lib/stream.ex:612: anonymous fn/4 in Stream.map/2
    (elixir 1.14.0) lib/stream.ex:1651: Stream.do_element_resource/6
Function: &Bonny.Server.AsyncStreamRunner.run/2
    Args: [#Stream<[enum: #Function<52.124013645/2 in Stream.resource/3>, funs: [#Function<48.124013645/1 in Stream.map/2>, #Function<48.124013645/1 in Stream.map/2>]]>, nil]

The postgresql-creds secret exists in the yggdrasil namespace.

This is because we recently changed the Operator's type from Cluster to Namespaced. That way Operator will only be able to list resources in its own namespace. To avoid this, you can create the secret of the access data to the database in the same namespace as the Operator, in this case eigr-functions

You need to define secret in eigr-functions namespace for now.

sleipnir · 2023-01-19T20:41:28Z

Resources managed by Operator will be created in the namespace you define. But the database secret needs to be in the eigr-functions namespace for now.

I haven't updated the documentation yet because I still don't know if this behavior will be maintained

janpieper · 2023-01-19T20:57:21Z

Having the database credentials stored in the operator namespace feels wrong, because sometimes you do not have access to it, especially when the operator is managed by another team than the application(s) that use the operator.

You need to define secret in eigr-functions namespace for now.

Sorry, I haven't seen your first reply 😅

sleipnir · 2023-01-19T20:57:33Z

In my machine:

sleipnir @ pop-os ~  
└─ $ (k8s: k3d-eigr-spawn) 🚀 ▶ k get system
No resources found in default namespace.
sleipnir @ pop-os ~  
└─ $ (k8s: k3d-eigr-spawn) 🚀 ▶ k apply -f /home/sleipnir/workspaces/eigr/spawn/examples/k8s/simple/system-1.yaml
actorsystem.spawn-eigr.io/test-1 created
sleipnir @ pop-os ~  
└─ $ (k8s: k3d-eigr-spawn) 🚀 ▶ k apply -f /home/sleipnir/workspaces/eigr/spawn/examples/k8s/simple/system-2.yaml
actorsystem.spawn-eigr.io/test-2 created
sleipnir @ pop-os ~  
└─ $ (k8s: k3d-eigr-spawn) 🚀 ▶ k get system
NAME     AGE
test-1   11s
test-2   4s
sleipnir @ pop-os ~  
└─ $ (k8s: k3d-eigr-spawn) 🚀 ▶ k apply -f /home/sleipnir/workspaces/eigr/spawn/examples/k8s/simple/host-1.yaml 
actorhost.spawn-eigr.io/spawn-springboot-example-1 created
sleipnir @ pop-os ~  
└─ $ (k8s: k3d-eigr-spawn) 🚀 ▶ k apply -f /home/sleipnir/workspaces/eigr/spawn/examples/k8s/simple/host-1.yaml 
actorhost.spawn-eigr.io/spawn-springboot-example-2 created
sleipnir @ pop-os ~  
└─ $ (k8s: k3d-eigr-spawn) 🚀 ▶ k get all
NAME                                              READY   STATUS                       RESTARTS         AGE
pod/spawn-springboot-example-1-b97dbb4bb-zrl62    1/2     CreateContainerConfigError   0                13s
pod/spawn-springboot-example-2-6cbfc6c89c-t7gxs   1/2     CreateContainerConfigError   0                3s

NAME                                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
service/kubernetes                       ClusterIP   10.43.0.1       <none>        443/TCP                               3h15m
service/spawn-springboot-example-svc     ClusterIP   10.43.226.235   <none>        8090/TCP,8091/TCP,9000/TCP,9001/TCP   155m
service/system-test-1                    ClusterIP   None            <none>        4369/TCP                              50s
service/system-test-2                    ClusterIP   None            <none>        4369/TCP                              43s
service/spawn-springboot-example-1-svc   ClusterIP   10.43.137.83    <none>        8090/TCP,8091/TCP,9000/TCP,9001/TCP   13s
service/spawn-springboot-example-2-svc   ClusterIP   10.43.38.46     <none>        8090/TCP,8091/TCP,9000/TCP,9001/TCP   3s

NAME                                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/spawn-springboot-example-1   0/1     1            0           13s
deployment.apps/spawn-springboot-example-2   0/1     1            0           3s

NAME                                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/spawn-springboot-example-1-b97dbb4bb    1         1         0       13s
replicaset.apps/spawn-springboot-example-2-6cbfc6c89c   1         1         0       3s

NAME                                                             REFERENCE                               TARGETS                        MINPODS   MAXPODS   REPLICAS   AGE
horizontalpodautoscaler.autoscaling/spawn-springboot-example-1   Deployment/spawn-springboot-example-1   <unknown>/70%, <unknown>/80%   1         8         0          13s
horizontalpodautoscaler.autoscaling/spawn-springboot-example-2   Deployment/spawn-springboot-example-2   <unknown>/70%, <unknown>/80%   1         8         0          3s

sleipnir @ pop-os ~  
└─ $ (k8s: k3d-eigr-spawn) 🚀 ▶ k get system
NAME     AGE
test-1   4m28s
test-2   4m21s
sleipnir @ pop-os ~  
└─ $ (k8s: k3d-eigr-spawn) 🚀 ▶

All the headless were created

sleipnir · 2023-01-20T01:39:17Z

@janpieper please use rc.12

janpieper · 2023-01-20T05:53:55Z

Are these IO.inspect() expected? 🤔

spawn/spawn_operator/spawn_operator/lib/spawn_operator/handler/actor_system_handler.ex

Line 40 in 651e29d

IO.inspect(resource, label: "Resource ---")
spawn/spawn_operator/spawn_operator/lib/spawn_operator/k8s/configmap/sidecar_configmap.ex

Line 20 in bb82f8d

IO.inspect(system, label: "System -----")
spawn/spawn_operator/spawn_operator/lib/spawn_operator/handler/actor_host_handler.ex

Line 45 in 651e29d

IO.inspect(resource, label: "Resource ---")

janpieper · 2023-01-20T07:51:46Z

Okay, I tried it with rc.12 and it looks a lot better 👍

$ kubectl get services -n yggdrasil
NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
[...]
system-yggdrasil     ClusterIP   None            <none>        4369/TCP                     24m

$ kubectl get services -n yggdrasil system-yggdrasil -o yaml | yq '.spec.selector'
actor-system: yggdrasil

sleipnir · 2023-01-20T12:50:22Z

Are these IO.inspect() expected? thinking

* https://github.com/eigr/spawn/blob/651e29dde442ba87b6298220d857703f6a6f2a10/spawn_operator/spawn_operator/lib/spawn_operator/handler/actor_system_handler.ex#L40

* https://github.com/eigr/spawn/blob/bb82f8dd00e51f523e00fe8d867edc77620053f8/spawn_operator/spawn_operator/lib/spawn_operator/k8s/configmap/sidecar_configmap.ex#L20

* https://github.com/eigr/spawn/blob/651e29dde442ba87b6298220d857703f6a6f2a10/spawn_operator/spawn_operator/lib/spawn_operator/handler/actor_host_handler.ex#L45

We are still in a pre-release version. The official launch will only take place in 0.5.0 in which we will have done a general cleaning of the code.

janpieper added the bug Something isn't working label Jan 19, 2023

sleipnir added enhancement New feature or request question Further information is requested operator and removed bug Something isn't working labels Jan 19, 2023

sleipnir mentioned this issue Jan 19, 2023

Remove unnecessary suffix in kubernetes resources. #124

Closed

sleipnir added bug Something isn't working and removed question Further information is requested labels Jan 19, 2023

sleipnir added a commit that referenced this issue Jan 19, 2023

Fix. This closes #124 #123

0b7f81c

janpieper closed this as completed Jan 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Wrong service name for actor system!? #123

Wrong service name for actor system!? #123

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023 •

edited

Loading

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023 •

edited

Loading

janpieper commented Jan 19, 2023

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023 •

edited

Loading

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

sleipnir commented Jan 19, 2023 •

edited

Loading

janpieper commented Jan 19, 2023 •

edited

Loading

sleipnir commented Jan 19, 2023

sleipnir commented Jan 20, 2023

janpieper commented Jan 20, 2023

janpieper commented Jan 20, 2023

sleipnir commented Jan 20, 2023

Wrong service name for actor system!? #123

Wrong service name for actor system!? #123

Comments

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023 • edited Loading

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023 • edited Loading

One ActorSystem

janpieper commented Jan 19, 2023

Two ActorSystems

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023 • edited Loading

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

janpieper commented Jan 19, 2023

sleipnir commented Jan 19, 2023

sleipnir commented Jan 19, 2023 • edited Loading

janpieper commented Jan 19, 2023 • edited Loading

sleipnir commented Jan 19, 2023

sleipnir commented Jan 20, 2023

janpieper commented Jan 20, 2023

janpieper commented Jan 20, 2023

sleipnir commented Jan 20, 2023

sleipnir commented Jan 19, 2023 •

edited

Loading

janpieper commented Jan 19, 2023 •

edited

Loading

janpieper commented Jan 19, 2023 •

edited

Loading

sleipnir commented Jan 19, 2023 •

edited

Loading

janpieper commented Jan 19, 2023 •

edited

Loading