Allow retrying on Service Unavailable errors from OpenAI

[agamble]

OpenAI is fairly unreliable in my experience and often returns ServiceUnavailable errors.

Eg:

• https://community.openai.com/t/i-have-the-error-openai-error-serviceunavailableerror-the-server-is-overloaded-or-not-ready-yet/290475
• https://community.openai.com/t/gpt-3-5-turbo-the-server-is-overloaded-or-not-ready-yet-errors/283196
• https://community.openai.com/t/continuous-server-overloaded-error-when-attempting-to-google-via-auto-gpt-paid-account/267802

This PR allows backoff and retry in response to these errors.

[MaartenGr]

Thank you for the PR! I just read through the links you provided and it seems that they did not specifically mention that this error could be directly resolved with backoff. Just to be sure, is the unreliability that you refer to something that should be fixed with backoff or is it something that temporarily happens within their servers? If it is the latter, then it might make unnecessary API calls as it will be fixed whenever their servers are not overloaded anymore.

[agamble]

Hey Maarten, thanks for the comment.

You're right that the links I shared don't specify that the errors are resolved via backoff, my intent in sharing them was to corroborate that many others are experiencing reliability issues with OpenAI - specifically with intermittent ServiceUnavailable errors. The links do say that these errors are due to OpenAI receiving too much traffic though, so a retry with backoff makes sense to me from that PoV.

Anecdotally, this change has completely resolve reliability issues I was seeing with using OpenAI during the representation stage. Prior to this change, it was extremely common to receive a ServiceUnavailable error from OpenAI, which would halt my entire clustering workflow.

[MaartenGr]

Apologies for the late reply!

Anecdotally, this change has completely resolve reliability issues I was seeing with using OpenAI during the representation stage. Prior to this change, it was extremely common to receive a ServiceUnavailable error from OpenAI, which would halt my entire clustering workflow.

Totally fair. Since your message/PR, there were also more users with this problem, so it definitely makes sense. When the pipeline runs, I'll make sure to merge this PR.