sciencemesh app does not select next client

[butonic]

the reva sciencemesh token.go only initializes a gateway client on service init:

type tokenHandler struct {
	gatewayClient    gateway.GatewayAPIClient
	meshDirectoryURL string
	providerDomain   string
	eventStream      events.Stream
}

func (h *tokenHandler) init(c *config) error {
	var err error
	h.gatewayClient, err = pool.GetGatewayServiceClient(c.GatewaySvc)
	if err != nil {
		return err
	}

	h.meshDirectoryURL = c.MeshDirectoryURL
	h.providerDomain = c.ProviderDomain

	if c.Events.Endpoint != "" {
		es, err := stream.NatsFromConfig("sciencemesh-token-handler", false, stream.NatsConfig(c.Events))
		if err != nil {
			return err
		}
		h.eventStream = es
	}

	return nil
}

this will cause requests to fail when the gateway service has moved to a diffferent ip

[butonic]

cc @d7oc

[d7oc]

Log error message:

{
  "level": "error",
  "service": "ocm",
  "pkg": "rhttp",
  "traceid": "45b32da09e0900e6645e4033f1375cf4",
  "request-id": "3d6dd949-cdd6-4c23-99a4-72f4fcb48004",
  "error": "rpc error: code = Unavailable desc = gateway: error calling FindAcceptedUsers: rpc error: code = Unavailable desc = last connection error: connection error: desc = \"transport: Error while dialing: dial tcp 10.42.0.108:9282: connect: no route to host\"",
  "time": "2024-09-16T15:53:55Z",
  "line": "github.com/cs3org/reva/v2@v2.24.1/internal/http/services/reqres/reqres.go:64",
  "message": "error sending a grpc find accepted users request"
}

[d7oc]

@butonic shouldn't this issue be on the infinite scale board?

[butonic]

pr in cs3org/reva#4851

[butonic]

landed in ocis with #10102