API Rate Limits & sleep_timer

[sh12111]

I'm currently using a "starter plan" API key from financialmodeling prep with toolkit however I seem to hit a rate limit when using a large ticker list (eg: 10k symbols):

companies = Toolkit(tickers=tickers, api_key=financetoolkitAPIkey, sleep_timer=True)
profiles = companies.get_profile()

Returns: "You have reached the rate limit of your subscription. This resulted in no data for"

I was under the impression that "sleep_timer" set to True would resolve such error?

I've been using the below as a workaround but requires some slight changes depending on data being pulled, eg get_historical_data()

chunks = [tickers[i:i + size] for i in range(0, len(tickers), size)]
    holder = pd.DataFrame()
    for i in range(0, len(chunks)):
        tickers = chunks[i]
        companies = Toolkit(tickers=tickers, api_key=financetoolkitAPIkey, start_date=start, end_date=stop)
        profiles = companies.get_profile()
        profiles = profiles.transpose()
        holder = pd.concat([holder, profiles])

        if len(chunks) > 1:
            print('sleeping for 65s to avoid API call / minute limit')
            time.sleep(65)

[JerBouma]

This is indeed how it should work, I need to look into the code again whether it correctly applies this timer. It could very well be that the error message changed which is why it nog longer reaches this part of the code: https://github.com/JerBouma/FinanceToolkit/blob/main/financetoolkit%2Fhelpers.py#L63

For historical data, this is because it will use Yahoo Finance if it fails to collect from FinancialModelingPrep, this may be undesirable but I've built it this way given that with e.g. a Free plan you can only get data from American exchanges and thus there is a fallback system in that case.

I test for the plan you are using, I can incorporate a setting which will override this behaviour from the historical data functionality. Thoughts?

[sh12111]

That would be most ideal for the historical data! Currently if it switches over to yahoo there's no error thrown so presumably one believes they've pulled all data when in fact there's missing data. thanks

[JerBouma]

@sh12111

Can you please verify that the tickers that returned no data, according to the message, actually have data? E.g. enter the ticker at the spot "TICKER_HERE" and your api key at the spot "API_KEY_HERE": https://financialmodelingprep.com/stable/profile?symbol=TICKER_HERE&apikey=API_KEY_HERE

I just looked into it and the sleep timer seems to work just fine, however if no data can be obtained it will still provide you with the following message:

    if limit_reach:
        print(
            "You have reached the rate limit of your subscription. This resulted in no "
            f"data for {', '.join(limit_reach)}\nIf you use the Free plan, consider "
            "upgrading your plan. You can get 15% off by using the following affiliate "
            "link which also supports the project: https://www.jeroenbouma.com/fmp"
        )

This is a confusing message since while it attempted to collect data for 60 seconds, it still failed which is most likely because the ticker simply had no data to begin with. I'll add a fix in the next release in which the message becomes more clear.

[sh12111]

I tried testing with the URL + my API key + ticker but I'm receiving:

{
  "Error Message": "Bandwidth Limit Reach . Please upgrade your plan or visit our documentation for more details at https://site.financialmodelingprep.com/"
}

Interestingly at https://site.financialmodelingprep.com/developer/docs/dashboard they provided an update on the changelog with the following:

Changes on February 24, 2025
API Bandwidth Limit Enabled : Bandwidth

API Keys over the bandwidth limit are now restricted due to high data transfer and downloads

I believe this is coincidental as I was running into the original issue over the last month but it appears I can't fully confirm until my data bandwidth consumption. Perhaps FMP rolled out the bandwidth limit earlier in the month and that's been causing the issue? I'll try to confirm by the 1st of the month.

[JerBouma]

Interesting! I'll ensure that this error message is included in the Finance Toolkit. This also makes it more relevant to be able to store a dataset locally and load it back into the Toolkit class given that you might only want to collect the data once in the first place, I'll work on that as well.

EDIT: Already built both in, working on the v2.0.0 release which will have all these features.

[JerBouma]

See: #176 :)