add initial delay seconds to brig liveness and readiness probes

[amitsagtani97]

Ticket - https://wearezeta.atlassian.net/browse/SQPIT-491

Brig pods continuously restarts in case of liveness and readiness probes failing on the startup in the low compute resource environment.

To reproduce -

• reduce the memory and cpu resource limits in charts/brig/values.yaml and deploy the chart.
• Components of brig will take time to start which is greater than the wait time for before running first liveness probe, hence the pod is marked as failed and restarted again and again.

Added a initial delay before running first liveness and readiness probe, which allows all the brig containers to start on a low resource environment before running the probes.

Checklist

• Add a new entry in an appropriate subdirectory of changelog.d
• Read and follow the PR guidelines

[jschaul]

I think your problem would be better solved with a startup probe, see also https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

startupProbe:
  httpGet:
   ...
   failureThreshold: 6
   periodSeconds: 5

That would wait for 30 seconds before moving over to the liveness probe which restarts brig.

Overall, if /i/status fails, however, then this begs the question if the installation works correctly otherwise? If brig doesn't have enough resources likely users will not have adequate latencies either.

Also, could it be that something else regarding networking is not working as it should, rather than this just being a resource problem?

[amitsagtani97]

Thank you, yes the startupProbe make more sense here, added it.

Yeah, there can be issues due to high network latencies as well, there is a argument "timeoutSeconds", which has default value of 1 second. Maybe, we can increase that as well for these probes.

[jschaul]

@amitsagtani97 PR looks good now; could you add one line to a changelog file (maybe under internal) then this PR is good to be (squash-) merged.