Continous retry on hitting rate limit

[ptrrrr]

The rate limit response code is 429 and is one of the retry code in

const RETRY_STATUS_CODES = [429, 500, 502, 503, 504];

So this is continous retry cycle

    } else if (RETRY_STATUS_CODES.includes(response.status)) {
      console.debug(
        `Retrying request on response status: ${response.status}`,
        `Response: ${await response.text()}`,
        `Attempt: ${attempts + 1}`,
      );
      // eslint-disable-next-line max-len
      await new Promise((resolve) =>
        setTimeout(resolve, Math.pow(2, (attempts + 1)) * 500),
      );
      
    I had like 50 requests in a few seconds.  The more requests I make, the more the rate limit is triggered.  
    
    So either we should change the response code for rate liits or ideally do a sliding window retry with extended delays.

[OlegIvaniv]

We're also running into this. On the website, you say the request limit is 5 per second, and the input limit is 2mil per minute. So if there are 20 requests, it should take ~ 4 seconds. Instead, it takes 150 seconds to complete all the retries. It almost seems like the retry requests are counting toward the rate limit

[sneko]

Since the team is not responding, I advise you to use something like patch-package (https://github.com/ds300/patch-package) to patch the package when installed (in your case, it would remove the 429 from the array). It saves time for sure 👍

[ptrrrr]

This is relatively easy fix in client.ts:
const RETRY_STATUS_CODES = [429, 500, 502, 503, 504];

remove 429

since 429 is a rate limiting response we should do a RETRY.

My only work around is using the python client

[GaspardBT]

Thanks for reporting this issue. We have deprecated this package in favor of mistralai/client-ts, which is the new official Mistral client, compatible with both TypeScript and JavaScript.

You can find all installation information here.

This change is effective starting with version 1.0.0 of the npm package.

Let us know if your issue persists with the new package by opening an issue there.