Add waitForReady to base config for FTR serverless

[Ikuni17]

Summary

waitForReady was added in #165467. This PR utilizes it so that FTR doesn't continue with starting Kibana until ES has reported its ready. We get quite a few of these errors every time, and it could be misleading. It adds 20-30s to startup time.

[mistic]

Other than the little comment I left, LGTM

[dmlemeshko]

Is it safe to say ES is ready when client.info() is returning result?
I tried client.cluster.health({ wait_for_status: 'green' }); and time frame is pretty the same, but it verifies ES cluster state is valid, e.g. "green" but not "yellow".

It looks like we already do health check for stateful https://github.com/elastic/kibana/blob/main/packages/kbn-es/src/cluster.js#L523-L560

[dmlemeshko]

@Ikuni17 I opened #165605 to unify wait logic across stateful & serverless as it seems like a valid check for both cases. I will try to convert cluster.js in TS separately so that it can also re-use waitForClusterReady.

Does it make sense?

[Ikuni17]

@dmlemeshko

Is it safe to say ES is ready when client.info() is returning result?
I tried client.cluster.health({ wait_for_status: 'green' }); and time frame is pretty the same, but it verifies ES cluster state is valid, e.g. "green" but not "yellow".

There are times where master node election can take longer, and keeps the cluster in yellow health. So, I think your solution is more appropriate for a true ready state. client.info returns basic information once the cluster is started, but does not necessarily mean healthy.

@Ikuni17 I opened #165605 to unify wait logic across stateful & serverless as it seems like a valid check for both cases. I will try to convert cluster.js in TS separately so that it can also re-use waitForClusterReady.

Should we combine the two PRs? We will need a bit of the client setup from this one. We decided to merge this now to help with pipeline stability and it should be easier to incorporate the changes into your PR. LMK if you need any help!

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: a812b31

Metrics [docs]

✅ unchanged

History

• 💔 Build #156604 failed c901ada
• 💔 Build #155856 failed 07b7e89

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @Ikuni17