get_nasdaq_symbols() is failing today. API endpoint problem?

[EternalNooblet]

I have been using this method daily for the last few weeks, so I know it worked on Friday. Below is the min code to cause the error. I am using python 3.11.3 and jupyter notebooks.

from pandas_datareader.nasdaq_trader import get_nasdaq_symbols
get_nasdaq_symbols()

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[1], line 2
      1 from pandas_datareader.nasdaq_trader import get_nasdaq_symbols
----> 2 get_nasdaq_symbols()

File ~/.local/lib/python3.11/site-packages/pandas_datareader/nasdaq_trader.py:107, in get_nasdaq_symbols(retry_count, timeout, pause)
    105 while retry_count > 0:
    106     try:
--> 107         _ticker_cache = _download_nasdaq_symbols(timeout=timeout)
    108         retry_count = -1
    109     except RemoteDataError:
    110         # retry on any exception

File ~/.local/lib/python3.11/site-packages/pandas_datareader/nasdaq_trader.py:70, in _download_nasdaq_symbols(timeout)
     65 # For pandas >= 0.20.0, the Python parser issues a warning if
     66 # both a converter and dtype are specified for the same column.
     67 # However, this measure is probably temporary until the read_csv
     68 # behavior is better formalized.
     69 with warnings.catch_warnings(record=True):
---> 70     data = read_csv(
     71         StringIO("\n".join(lines[:-1])),
     72         "|",
     73         dtype=_TICKER_DTYPE,
     74         converters=converter_map,
     75         index_col=1,
     76     )
     78 # Properly cast enumerations
     79 for cat in _CATEGORICAL:

TypeError: read_csv() takes 1 positional argument but 2 positional arguments (and 3 keyword-only arguments) were given

[Wade0125Studio]

`import pandas_datareader.data as web

import pandas as pd

import datetime as dt

df = web.DataReader('GE', 'yahoo', start='2019-09-10', end='2019-10-09')

df.head() ---------------------------------------------------------------------------`
TypeError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_12248\1576652786.py in
5 import datetime as dt
6
----> 7 df = web.DataReader('GE', 'yahoo', start='2019-09-10', end='2019-10-09')
8
9 df.head()

~\anaconda3\envs\Machine-Learning-Environment\lib\site-packages\pandas\util_decorators.py in wrapper(*args, **kwargs)
209 else:
210 kwargs[new_arg_name] = new_arg_value
--> 211 return func(*args, **kwargs)
212
213 return cast(F, wrapper)

~\anaconda3\envs\Machine-Learning-Environment\lib\site-packages\pandas_datareader\data.py in DataReader(name, data_source, start, end, retry_count, pause, session, api_key)
368
369 if data_source == "yahoo":
--> 370 return YahooDailyReader(
371 symbols=name,
372 start=start,

~\anaconda3\envs\Machine-Learning-Environment\lib\site-packages\pandas_datareader\base.py in read(self)
251 # If a single symbol, (e.g., 'GOOG')
252 if isinstance(self.symbols, (string_types, int)):
--> 253 df = self._read_one_data(self.url, params=self._get_params(self.symbols))
254 # Or multiple symbols, (e.g., ['GOOG', 'AAPL', 'MSFT'])
255 elif isinstance(self.symbols, DataFrame):

~\anaconda3\envs\Machine-Learning-Environment\lib\site-packages\pandas_datareader\yahoo\daily.py in _read_one_data(self, url, params)
151 try:
152 j = json.loads(re.search(ptrn, resp.text, re.DOTALL).group(1))
--> 153 data = j["context"]["dispatcher"]["stores"]["HistoricalPriceStore"]
154 except KeyError:
155 msg = "No data fetched for symbol {} using {}"

TypeError: string indices must be integers

[patpatwithhat]

I have been using this method daily for the last few weeks, so I know it worked on Friday. Below is the min code to cause the error. I am using python 3.11.3 and jupyter notebooks.

from pandas_datareader.nasdaq_trader import get_nasdaq_symbols
get_nasdaq_symbols()

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[1], line 2
      1 from pandas_datareader.nasdaq_trader import get_nasdaq_symbols
----> 2 get_nasdaq_symbols()

File ~/.local/lib/python3.11/site-packages/pandas_datareader/nasdaq_trader.py:107, in get_nasdaq_symbols(retry_count, timeout, pause)
    105 while retry_count > 0:
    106     try:
--> 107         _ticker_cache = _download_nasdaq_symbols(timeout=timeout)
    108         retry_count = -1
    109     except RemoteDataError:
    110         # retry on any exception

File ~/.local/lib/python3.11/site-packages/pandas_datareader/nasdaq_trader.py:70, in _download_nasdaq_symbols(timeout)
     65 # For pandas >= 0.20.0, the Python parser issues a warning if
     66 # both a converter and dtype are specified for the same column.
     67 # However, this measure is probably temporary until the read_csv
     68 # behavior is better formalized.
     69 with warnings.catch_warnings(record=True):
---> 70     data = read_csv(
     71         StringIO("\n".join(lines[:-1])),
     72         "|",
     73         dtype=_TICKER_DTYPE,
     74         converters=converter_map,
     75         index_col=1,
     76     )
     78 # Properly cast enumerations
     79 for cat in _CATEGORICAL:

TypeError: read_csv() takes 1 positional argument but 2 positional arguments (and 3 keyword-only arguments) were given

same problem for me!

[KamarajuKusumanchi]

I found that it works with (pandas 1.5.3, pandas-datareader 0.10.0) but not with (pandas 2.0.3, pandas-datareader 0.10.0).

Consider the following environment files

 % cat env_test_pandas_153.yml
name: test_pandas_153
channels:
  - defaults
dependencies:
  - python=3.11
  - pandas=1.5.3
  - pandas-datareader=0.10.0

 % cat env_test_pandas_203.yml
name: test_pandas_203
channels:
  - defaults
dependencies:
  - python=3.11
  - pandas=2.0.3
  - pandas-datareader=0.10.0

Create the respective environments with

 % conda env create -f env_test_pandas_153.yml
 % conda env create -f env_test_pandas_203.yml

Sample code works with test_pandas_153 environment

 % conda activate test_pandas_153             

 % which python
/opt/rajulocal/miniconda3/envs/test_pandas_153/bin/python

 % python
Python 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pandas_datareader.nasdaq_trader import get_nasdaq_symbols
>>> traded_symbols = get_nasdaq_symbols()
>>> traded_symbols.shape
(11321, 11)

 % conda deactivate

The same code does not work with test_pandas_203 environment

 % conda activate test_pandas_203

 % which python
/opt/rajulocal/miniconda3/envs/test_pandas_203/bin/python

 % python
Python 3.11.5 (main, Sep 11 2023, 13:54:46) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pandas_datareader.nasdaq_trader import get_nasdaq_symbols
>>> traded_symbols = get_nasdaq_symbols()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/rajulocal/miniconda3/envs/test_pandas_203/lib/python3.11/site-packages/pandas_datareader/nasdaq_trader.py", line 107, in get_nasdaq_symbols
    _ticker_cache = _download_nasdaq_symbols(timeout=timeout)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rajulocal/miniconda3/envs/test_pandas_203/lib/python3.11/site-packages/pandas_datareader/nasdaq_trader.py", line 70, in _download_nasdaq_symbols
    data = read_csv(
           ^^^^^^^^^
TypeError: read_csv() takes 1 positional argument but 2 positional arguments (and 3 keyword-only arguments) were given

 % conda deactivate

[opnagroup]

I'm facing this issue too.

How would one go about correcting either the get_nasdaq_symbols() or the read_csv() scripts to make this work again?

[KamarajuKusumanchi]

You can fix the error locally as follows:

In pandas_datareader/nasdaq_trader.py, change

    with warnings.catch_warnings(record=True):
        data = read_csv(
            StringIO("\n".join(lines[:-1])),
            "|",
            dtype=_TICKER_DTYPE,
            converters=converter_map,
            index_col=1,
        )

to

    with warnings.catch_warnings(record=True):
        data = read_csv(
            StringIO("\n".join(lines[:-1])),
            sep="|",
            dtype=_TICKER_DTYPE,
            converters=converter_map,
            index_col=1,
        )

That is explicitly specify the parameter name ("sep") when calling read_csv.

FWIW, @FidoDido1982 raised a pull request https://github.com/pydata/pandas-datareader/pull/968/files for this and it is merged into the main branch. If and when a new version of pandas-datareader is released we should be good, I think.

[opnagroup]

Wow, thank you so much. This fixed it!