Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Response format from Yahoo seems to have changed I keep getting this error. #952

Open
yeison opened this issue Dec 17, 2022 · 98 comments · May be fixed by #953
Open

Response format from Yahoo seems to have changed I keep getting this error. #952

yeison opened this issue Dec 17, 2022 · 98 comments · May be fixed by #953

Comments

@yeison
Copy link

yeison commented Dec 17, 2022

File "/Users/yeison/miniforge3/envs/tf-metal-0.6.0/lib/python3.10/site-packages/pandas_datareader/yahoo/daily.py", line 153, in _read_one_data
data = j["context"]["dispatcher"]["stores"]["HistoricalPriceStore"]
TypeError: string indices must be integers

@DevAleksei
Copy link

As work around you may try to use solution from https://pypi.org/project/yfinance/
title pandas_datareader override

@ivofernandes
Copy link

They started to encrypt the stores, looks like they encrypted with AES or something like that and now stores values comes as a string, not sure how to solve that

@raphi6
Copy link

raphi6 commented Dec 17, 2022

Is there some way that I can implement a temporary solution using yfinance as an override and pull request it ?

@DenisOd91
Copy link

Is there some way that I can implement a temporary solution using yfinance as an override and pull request it ?

>>> from pandas_datareader import data as pdr
>>> import yfinance as yf
>>> yf.pdr_override()
>>> y_symbols = ['SCHAND.NS', 'TATAPOWER.NS', 'ITC.NS']
>>> from datetime import datetime
>>> startdate = datetime(2022,12,1)
>>> enddate = datetime(2022,12,15)
>>> data = pdr.get_data_yahoo(y_symbols, start=startdate, end=enddate)

Source on StackOverflow

@raphi6
Copy link

raphi6 commented Dec 17, 2022

Is there some way that I can implement a temporary solution using yfinance as an override and pull request it ?

>>> from pandas_datareader import data as pdr
>>> import yfinance as yf
>>> yf.pdr_override()
>>> y_symbols = ['SCHAND.NS', 'TATAPOWER.NS', 'ITC.NS']
>>> from datetime import datetime
>>> startdate = datetime(2022,12,1)
>>> enddate = datetime(2022,12,15)
>>> data = pdr.get_data_yahoo(y_symbols, start=startdate, end=enddate)

Source on StackOverflow

Oh yeah I have used this but I meant fixing pandas-datareader as I have already submitted the code for my dissertation so I can't change my code directly, the only way I can change my code is by changing the library hehe. sorry for confusion

@raphi6
Copy link

raphi6 commented Dec 18, 2022

They started to encrypt the stores, looks like they encrypted with AES or something like that and now stores values comes as a string, not sure how to solve that

What are the stores? I will try anything to fix this as I have just submitted my interim dissertation and without this library, my code has no data :/ due to the nature of academia I dont think asking if the marker can override with yfinance will work

@DevAleksei
Copy link

DevAleksei commented Dec 18, 2022

What are the stores? I will try anything to fix this as I have just submitted my interim dissertation and without this library, my code has no data :/ due to the nature of academia I dont think asking if the marker can override with yfinance will work

Then you are the best candidate to fix this library.
j["context"]["dispatcher"]["stores"] response contains some encrypted data instead of plain object, so you may take a look at how this data is handled in yfinance, and migrate the code. Good luck.

@raphi6
Copy link

raphi6 commented Dec 18, 2022

What are the stores? I will try anything to fix this as I have just submitted my interim dissertation and without this library, my code has no data :/ due to the nature of academia I dont think asking if the marker can override with yfinance will work

Then you are the best candidate to fix this library. j["context"]["dispatcher"]["stores"] response contains some encrypted data instead of plain object, so you may take a look at how this data is handled in yfinance, and migrate the code. Good luck.

With a stroke of magical programmer luck I have managed to migrate some code from yfinance where now the data is decrypted and I have tested on a small number of stocks I am able to get their data.

         I changed code in: pandas_datareader/yahoo/daily.py   Just adding a decryption function and making sure data is 
         handled correctly like before .

It took me all day as I have never worked on such a professional code base so far.
But that leads to another issue now, How do I go about merging my solution or I guess I have to create a pull request now? I am not sure how this works!

pandas-datareader was checkedout locally to my IDE and I have made all the changes there, what do you suggest me to do? I am sure there are still things needed to be added to the codebase like requirements etc? idk this is my first time.

Thanks for any help.

@raphi6
Copy link

raphi6 commented Dec 18, 2022

File "/Users/yeison/miniforge3/envs/tf-metal-0.6.0/lib/python3.10/site-packages/pandas_datareader/yahoo/daily.py", line 153, in _read_one_data data = j["context"]["dispatcher"]["stores"]["HistoricalPriceStore"] TypeError: string indices must be integers

I have found a fix, as stated in above comment. Do you know how about I will go creating a pull request? I just tried on my IDE and got permission denied? Any suggestions?

@mazalkov
Copy link

@raphi6 have you tried branching from the downloaded repository and then pushing your changes? It should say there's no upstream branch, by setting one you'll create a pull request.

@raphi6
Copy link

raphi6 commented Dec 18, 2022

Ahh yes branching, I always forget. So now I will checkout pandas-datareader, create a branch and add my changes, create a pull request. And that should work?

@mazalkov
Copy link

Yes I think so, it may not be the best way of doing it but I do not currently know of a better one. By setting an upstream branch and pushing your branch to it, it should create the PR.

@raphi6
Copy link

raphi6 commented Dec 18, 2022

Yes I think so, it may not be the best way of doing it but I do not currently know of a better one. By setting an upstream branch and pushing your branch to it, it should create the PR.

image

I just tried creating a new branch and commiting, pushing, then creating a PR and get the above, What did you mean by setting an upstream branch?

@mazalkov
Copy link

You may be trying to push directly to master, which only the repo owners will have permission for (although it is advised never to push directly to master in any case). To not take up replies in the thread, I would recommend looking into the process of opening a PR on GitHub for another repo using online resources:

Link 1
Link 2
LInk 3

Apologies I can't be any more help, quite new to Git in terms of contributing to 3rd party repos.

@raphi6
Copy link

raphi6 commented Dec 19, 2022

I've opened a pull request with working code, be sure to check it out if it works for you guys.

@joanlofe
Copy link

Hi, @raphi6 , I can confirm it works perfectly fine. Thanks a lot for the quick fix! Let's see if they approve the pull request.

For reference, I cloned your pull request and installed it using the following sequence of commands (in Ubuntu 20.04):

git clone https://github.com/raphi6/pandas-datareader.git
git checkout 'Yahoo!_Issue#952'
conda uninstall pandas-datareader
conda install pycryptodome pycryptodomex
python setup.py install --record installed_files.txt

The --record argument in the install command is to get a list of installed files, so that it is easy to uninstall in the future (following this SO thread). The pycrypto* files are dependencies I has to install to make it work.

@pyproper
Copy link

Hi, @raphi6 , I can confirm it works perfectly fine. Thanks a lot for the quick fix! Let's see if they approve the pull request.

For reference, I cloned your pull request and installed it using the following sequence of commands (in Ubuntu 20.04):

git clone https://github.com/raphi6/pandas-datareader.git
git checkout 'Yahoo!_Issue#952'
conda uninstall pandas-datareader
conda install pycryptodome pycryptodomex
python setup.py install --record installed_files.txt

The --record argument in the install command is to get a list of installed files, so that it is easy to uninstall in the future (following this SO thread). The pycrypto* files are dependencies I has to install to make it work.

Can this be installed or implemented on Google Colabs?

@HiroshiOkada
Copy link

The following seems to work well on Google Colab.

%%shell
git clone https://github.com/raphi6/pandas-datareader.git
cd pandas-datareader 
git checkout 'Yahoo!_Issue#952'
pip install pycryptodome pycryptodomex
python setup.py install --record installed_files.txt

@pyproper
Copy link

pyproper commented Dec 20, 2022

The following seems to work well on Google Colab.

%%shell
git clone https://github.com/raphi6/pandas-datareader.git
cd pandas-datareader 
git checkout 'Yahoo!_Issue#952'
pip install pycryptodome pycryptodomex
python setup.py install --record installed_files.txt

/usr/local/lib/python3.8/dist-packages/pandas_datareader/base.py:272: SymbolWarning: Failed to read symbol:
RemoteDataError: No data fetched using 'YahooDailyReader'
I get this which started to be an issue a year or so ago and have been using this since then pip install git+https://github.com/pydata/pandas-datareader.git
Is it possible to put them together?

@nichaelwichterle1
Copy link

Hello - I've got a few complex scripts running in jupiter notebook pulling data from yahoo finance - what are the exact commands which need to be entered in order to regain access so that I dont get the "string indices must be integers" error ?
much appreciated, thx for the help

@joanlofe
Copy link

joanlofe commented Dec 20, 2022

The following seems to work well on Google Colab.

%%shell
git clone https://github.com/raphi6/pandas-datareader.git
cd pandas-datareader 
git checkout 'Yahoo!_Issue#952'
pip install pycryptodome pycryptodomex
python setup.py install --record installed_files.txt

/usr/local/lib/python3.8/dist-packages/pandas_datareader/base.py:272: SymbolWarning: Failed to read symbol: RemoteDataError: No data fetched using 'YahooDailyReader' I get this which started to be an issue a year or so ago and have been using this since then pip install git+https://github.com/pydata/pandas-datareader.git Is it possible to put them together?

@pyproper Yes, it is. You can use this:

 pip install git+https://github.com/raphi6/pandas-datareader.git@ea66d6b981554f9d0262038aef2106dda7138316

Notice I am using the commit hash here instead of a branch name, because it is Yahoo!_Issue#952 and there is an issue with hash characters when using pip this way.

@nichaelwichterle1
Copy link

Thanks very much, all looks good now!

@pyproper
Copy link

@pyproper Yes, it is. You can use this:

 pip install git+https://github.com/raphi6/pandas-datareader.git@ea66d6b981554f9d0262038aef2106dda7138316

Notice I am using the commit hash here instead of a branch name, because it is Yahoo!_Issue#952 and there is an issue with hash characters when using pip this way.

Perfect! Thank You

@KryptoEmman
Copy link

@joanlofe

@pyproper Yes, it is. You can use this:

 pip install git+https://github.com/raphi6/pandas-datareader.git@ea66d6b981554f9d0262038aef2106dda7138316

Notice I am using the commit hash here instead of a branch name, because it is Yahoo!_Issue#952 and there is an issue with hash characters when using pip this way.

I had to use pip3 to install, but now have the following error:

Traceback (most recent call last):
File "USP.py", line 6, in
from pandas_datareader import data as pdr
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas_datareader/init.py", line 5, in
from .data import (
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas_datareader/data.py", line 11, in
from pandas_datareader.av.forex import AVForexReader
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas_datareader/av/init.py", line 5, in
from pandas_datareader._utils import RemoteDataError
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas_datareader/_utils.py", line 6, in
from pandas_datareader.compat import is_number
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas_datareader/compat/init.py", line 1, in
from packaging import version
ModuleNotFoundError: No module named 'packaging'

Thoughts?

@joanlofe
Copy link

@KryptoEmman Have you tried pip install packaging and then repeat again?

@CKDarling
Copy link

CKDarling commented Dec 21, 2022

I've opened a pull request with working code, be sure to check it out if it works for you guys.

This fix by @raphi6 needs to be made a priority. Yahoo API is bricked through PDR.

@raphi6
Copy link

raphi6 commented Dec 22, 2022

I've opened a pull request with working code, be sure to check it out if it works for you guys.

This fix by @raphi6 needs to be made a priority. Yahoo API is bricked through PDR.

Thank you! if anyone could get into contact with someone that can accept the PR, that would be great!

@nichaelwichterle1
Copy link

Thanks very much everyone! I've migrated to yfinance and am looking to unplug/replug the data now from yfinance into my originally designed scripts. The only problem I seem to be having now relates to Dates: the format i want is simply year-month-date but I'm also getting a timestamp. I tried the formats above to no avail. Using widgets, some of the errors I've been encountering have been:
1 Failed download:

  • TICKER1: No timezone found, symbol may be delisted

I'm using widgets for tickers and timeframes:
)
startDate = widgets.DatePicker(
value=datetime.date(2010,1,1),
description='Start Date',
disabled=False
)
endDate = widgets.DatePicker(
value=datetime.date.today(),
description='End Date',
disabled=False
)

any input would be much appreciated! many thanks

@nichaelwichterle1
Copy link

stock1 = yf.download("ticker1", start="startDate", end="endDate")

@nichaelwichterle1
Copy link

nichaelwichterle1 commented Jan 17, 2023

Or specifically, the date I'm getting in my dataframe is: 2022-02-17 00:00:00-05:00 Would anyone know how to get rid of the time component ? All I need is the date 2022-02-17 and not the 00:00:00-05:00 component

I think that's why I'm getting getting a "TypeError: float() argument must be a string or a number, not 'Timestamp'
Thanks again everyone! much appreciated

@nmaiorana
Copy link

nmaiorana commented Jan 17, 2023

After you get the data, you need to change the index, it's not interpreted as a date from yfinance.

I had to convert mine to Pandas DatetimeIndex.

    price_histories = yf.download(tickers=symbols, auto_adjust=True, **argv)
    price_histories.index = pd.DatetimeIndex(price_histories.index)

This does not get rid of the time component, but will allow you to filter by date.

@CKDarling
Copy link

As work around you may try to use solution from https://pypi.org/project/yfinance/ title pandas_datareader override

Im going on record and saying all who use Yahoo! Finance data need to put effort into maintaining Yfinance. This issue has been apparent for over a month and very little has been done by the team who manages this repo. Yfinance admin fixed it within a few days of the Yahoo! API encryption.

@nichaelwichterle1
Copy link

nichaelwichterle1 commented Jan 18, 2023

After you get the data, you need to change the index, it's not interpreted as a date from yfinance.

I had to convert mine to Pandas DatetimeIndex.

    price_histories = yf.download(tickers=symbols, auto_adjust=True, **argv)
    price_histories.index = pd.DatetimeIndex(price_histories.index)

This does not get rid of the time component, but will allow you to filter by date.

Thanks very much for your help, pls excuse my lack of programing proficiency - I feel like I'm on the verge of getting this all done but just not quite there yet. I'm using:

ticker1 = widgets.Text(
value='TSLA',
description='Ticker 1',
placeholder='TSLA',
disabled=False
)
startDate = widgets.DatePicker(
value=datetime.date(2022,2,17),
description='Start Date',
disabled=False
)
endDate = widgets.DatePicker(
value=datetime.date.today(),
description='End Date',
disabled=False
)
Stock1 = yf.download(ticker1.value, start=startDate.value, end=endDate.value)

...this all works so far however I'm still getting Timestamp data and have not been able to get a string or number as the rest of my code would need. Where would I insert the above two lines you highlighted above ? I've been trying to convert the date index
I'm sorry, still struggling with this one...

@NRHartFRC
Copy link

NRHartFRC commented Jan 18, 2023

Interpreted conflict summary & progress breakdown:

Appears Yahoo Finance (YF) has migrated their codebase to a different architecture--perhaps react+redux architecture--which is rendering API objects/variables/data in a non-conventional manner. Raphi6 has decrypted the YF [AES] feature and has modified the pandas-datareader library accordingly. Now, we are experiencing data acquisition bugs (e.g., string indicies must be integers, new_j is referenced before assignment, etc...).

Proposed crux:

React-Redux architecture operates as a real-time, 'global state bus' to hold all variables/objects/states/values/facts/truths, and updates them without requiring end-user intervention between updates [1]. Long term scalability is the motivation here for YF. "Why is local context required at all?," is the rhetoric that contradicts our understanding of this migration bug. Per Figure 1, while actions/events from the UI are dispatched to a reducer (located in the store), the actions undergo a 'state' interaction--depicted by a state 'loopback.' Once reduced, this interaction event is seamlessly updated in the store (globally). I believe this interaction phase (between reducer and store) is casting the datatype to a string element array (despite pdr wanting an integer to start with), plus this is happening all out of context (pdr wants local context, YF uses global state context). The spread operator [...], located in the reducer component (.js file on YF backend) for YF HistoricalPriceStore encryption, creates a 'shallow clone' of the original object, with global-context, and string datatype of all dispatched events/actions. Once the global store 'new_j' state is updated (that is an action that is dispatched to the store on YF), the 'local-context-state' of new_j has not yet been set as a local reference yet--only as a global reference in the redux store. The state of the redux store is updated using the spread operator, in a global context, as a string, before the python compiler interprets the locally referenced decryption assignment in your daily.py component (python/pdr wants integers). So fixing one problem, say the datatype issue, creates another migration issue by virtue of Redux dataflow (see Figure 1).

Proposed solution:

Curly braces {} could be beneficial to help python see a 'key-value pair' from JSX parser on YF backend (see event: [action.state, ...state.events] below). In other words, a dictionary could be created, rather than malformed string that is fetched from redux store to create new_j (i.e., malformed for pdr). Attribute key is the decrypted new_j state, value is the decrypted HistoricalPriceStore value. So, change your 'event handling' decryption process in daily.py. Specifically that of the data = new_j['HistoricalPriceStore']. Pandas-datareader appears to want a numerical datatype for new_j rather than a string, YF wants string to talk to the redux store. Try updating your event attribute handling of the new_j data list as a dictionary. Specifically, try experimenting with curly braces in combination with the grave symbol, or backquote (located above tab key) to embed integers into strings, then parse as needed. The event on YF backend will dispatch the 'action.event' (current array state, interpreted as local to your event handler function read_one_data). In the store, '...state.events' (appended state update request/confirmation, interpreted as global event handler action that fetches prices from HistoricalPriceStore list) needs to wait for the python reference to be acknowledged before updating. Fetch HistoricalPriceStore from a dictionary that has the proper typecasting and context.Theoretically, new_j bug could be a matter of object/variable --> (redux-state) declaration/assignment vs. re-declaration/re-assignment that is out of context. Solution is to pull the string element values from the HistoricalPriceStore event, typecast the values to integers in a local context for python/pdr to use, while avoiding emitting an event/action to the redux store via YF dispatcher (this happens in parallel, which is what gives UnboundLocalError). Again, this must happen before the redux state is updated in the global-context-store, otherwise new_j will be out of context. Mediocre solution would involve making 'new_j' a global variable, so as to mimic a redux store state that may be accessible at a higher level, and more importantly, in the proper global context (will likely not work). Spread operator (enemy here) ought to appear as an attribute key-value pair in a case statement in the reducers.js component (of YF backend) and may appear as such:

Sample YF backend key-value attribute handling for new_j data action event to update HistoricalPriceStore (all YF backend):

events: [action.event, ...state.events] //current key-value pair in redux (data assignment), update store events

Sample reference case statement REDUCER for react-redux (typ. JSX syntax rather than javascript):

case 'DECYRPTION_SUCCESS':
return {
...state,
decryption: {
decryptionType: 'AES',
isPending: false,
isSuccessful: true
},
decryptionInProgress: false,
events: [action.event, ...state.events] //HistoricalPriceStore decryption, ...append decrypted value to store
}

Disclaimer: I am not an expert on this subject. Hope this helps your dissertation success. All for OP and OP for all!

let
Spread operator = [...] appends elements to existing array, while preserving original array (seems to be a string here)
Dispatch = construct action/payload event for global state acquisition (declaration, or redeclaration)
Event handler = arbitrary function for data acquisition, function called for UI actions
Event = message/attribute in reducers.js component file and top-level function (i.e., deposit handler event [1])
Emit = dispatch/broadcast an event
Bus = binary registry stack of attributes/events
Component = javascript (JSX) file in /src/assets directory
State = binary representation of facts, truth, objects, variables, results, booleans, etc...
Action = input at UI level, string of attributes bundled into an 'event' that is dispatched to reducer, then to store
Reducer = data handler (de-abstraction step, metaphorical automated binary interface that talks to the store)
Interactions = cross-functional/cross-app data acquisition from previous 'state' to new 'state'
Store = truth/fact/global state bus/message bank/database/

Figure 1. Redux dataflow https://redux.js.org/tutorials/fundamentals/part-2-concepts-data-flow [1]

@nichaelwichterle1
Copy link

nichaelwichterle1 commented Jan 19, 2023

Or can anyone provide some online forum where I can seek some help and ask some python related questions ? I really think I'm close but would love to get this to work again. I'd even pay for a bit of help!
Much appreciated

@santosh1383
Copy link

Here I share a few tricks to solve this problem:

import pandas
from pandas_datareader import data as pdr
import yfinance as yfin
yfin.pdr_override()

then

df = pdr.get_data_yahoo("TSLA", start="yyyy-mm-dd", end="yyyy-mm-dd")
print(df)

Hopefully it can be useful, thanks :)

@nichaelwichterle1
Copy link

nichaelwichterle1 commented Jan 20, 2023

Thanks very much!! ... I'm using the yfin.pdr_override() but still one little issue however:

Stock1 = pdr.get_data_yahoo("ticker1", start="yyyy-mm-dd", end="yyyy-mm-dd")
Stock1

now the error I'm getting is:
1 Failed download:

  • TICKER1: No timezone found, symbol may be delisted

btw when I type in: Stock1.index.dtype --> datetime64[ns, America/Toronto]

@nichaelwichterle1
Copy link

Thanks again everyone for your help, unfortunately I will be away all of next week but hopefully when I'm back and can give another crack at this (given all your input) I'll finally get things to work again (fingers crossed). Once again, thx very much in advance. Back in a week!

@davimmilhome
Copy link

This issue still up, but im very impressed with your tryes to solve it

@yuzhipeter
Copy link

yahoo finance has new problem again.

df = web.DataReader("SPY", data_source='yahoo', start='2023-01-01', end='2023-01-13', session = sesh)

Error msg

Traceback (most recent call last):

File "C:\Users\peter\AppData\Local\Temp/ipykernel_27516/630144412.py", line 1, in
df = web.DataReader("SPY", data_source='yahoo', start='2023-01-01', end='2023-01-13', session = sesh)

File "C:\Users\peter\anaconda3\lib\site-packages\pandas\util_decorators.py", line 207, in wrapper
return func(*args, **kwargs)

File "C:\Users\peter\anaconda3\lib\site-packages\pandas_datareader\data.py", line 370, in DataReader
return YahooDailyReader(

File "C:\Users\peter\anaconda3\lib\site-packages\pandas_datareader\base.py", line 253, in read
df = self._read_one_data(self.url, params=self._get_params(self.symbols))

File "C:\Users\peter\anaconda3\lib\site-packages\pandas_datareader\yahoo\daily.py", line 227, in _read_one_data
new_j = decrypt_cryptojs_aes(

File "C:\Users\peter\anaconda3\lib\site-packages\pandas_datareader\yahoo\daily.py", line 81, in decrypt_cryptojs_aes
plaintext = unpad(plaintext, 16, style="pkcs7")

File "C:\Users\peter\anaconda3\lib\site-packages\Crypto\Util\Padding.py", line 92, in unpad
raise ValueError("Padding is incorrect.")

ValueError: Padding is incorrect.

Thanks!

@pyproper
Copy link

I just asked ChatGPT and it suggested using yfinance. Has anyone had experience with it? The data format looks the same.

price_histories = yf.download(tickers=stock_universe,
                                  period='5y',
                                  auto_adjust=True)

So far no issues except I had to get the latest version to match my pandas version.

From Chat GPT:

import yfinance as yf
import datetime

# Define the ticker symbol and date range
ticker = "AAPL"
start = datetime.datetime.now() - datetime.timedelta(days=365)
end = datetime.datetime.now()

# Download the historical stock prices
data = yf.download(ticker, start=start, end=end)

# Print the data
print(data)

Thank you @nmaiorana
Just wanted to confirm this works as of 1/26/23 13:28 pm
I only pulled prices so I cannot confirm that other data attributes work or not.

@jfomann29
Copy link

@nichaelwichterle1 - try this after creation of data frame:

df['Date_New'] = data.index
df['Date_New'] = pd.to_datetime(df['Date_New']).tz_convert('America/New_York').dt.strftime('%Y-%m-%d')
df = df.set_index('Date_New')

@nichaelwichterle1
Copy link

Thank you so much, after much trying I'm finally getting an output and all works for Stock1 and for that I'm overjoyed!

My last question however is that I need to do the same process for Stock2, however the output for Stock2 seems to be a float. How can I convert Stock2 to a dt format like Stock1 so that I can run the same process and then finally graph both 1 and 2 ?
Once again, I'm very appreciative of all the help!

Python Screenshot 2023-01-30new

@nichaelwichterle1
Copy link

Hi everyone, Im making alot of progress and really getting close now – can someone pls help me with my code, the output I want is almost all there but I’d love to know what code I should insert so that the dates on the x-axis are automatically spaced out depending on the date range I input.

The pic below is what I’m getting so far, as you can see, the spacing of the dates is problematic. That is the last issue I’m having.
Many thanks for your help and input, it is all much appreciated!!
Final output for dscrd

@nichaelwichterle1
Copy link

Right, I am using yfinance to pull the data. I just am not able to change the date spacing on the x axis however. Anyone ?

@jfomann29
Copy link

@nichaelwichterle1 what data type is the index of your underlying dataframe? If it is not DatetimeIndex, try that before plotting the graph.

import datetime as dt
df.index = pd.to_datetime(df.index)

@nichaelwichterle1
Copy link

Thanks very much for your input. I was initially having problems because downloading the dataframe initially yielded year-month-day-hour-seconds which was problematic. I had to convert it via:
Stock1['Date_New'] = pd.to_datetime(Stock1['Date_New']).tz_convert('America/New_York').dt.strftime('%Y-%m-%d')

this worked well enough but I have no idea how to change the date spacing on the x axis.
Thx again, it is very much appreciated!

@uad1098
Copy link

uad1098 commented Mar 15, 2023

when I code yfinance as follows by santosh1383 commented on Jan 20

import pandas
from pandas_datareader import data as pdr
import yfinance as yfin
yfin.pdr_override()

df = pdr.get_data_yahoo("TSLA", start="yyyy-mm-dd", end="yyyy-mm-dd")
print(df)

I get exception following exception error. Can anyone tell me why and what am I missing?
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Users\RAM LLC\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "C:\Users\RAM LLC\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 870, in run
self._target(*self._args, **self.kwargs)
File "C:\Users\RAM LLC\AppData\Local\Programs\Python\Python38-32\lib\site-packages\multitasking_init
.py", line 102, in _run_via_pool
return callee(*args, **kwargs)
File "C:\Users\RAM LLC\AppData\Local\Programs\Python\Python38-32\lib\site-packages\yfinance\multi.py", line 169, in _download_one_threaded
data = _download_one(ticker, start, end, auto_adjust, back_adjust,
File "C:\Users\RAM LLC\AppData\Local\Programs\Python\Python38-32\lib\site-packages\yfinance\multi.py", line 181, in download_one
return Ticker(ticker).history(period=period, interval=interval,
File "C:\Users\RAM LLC\AppData\Local\Programs\Python\Python38-32\lib\site-packages\yfinance\base.py", line 157, in history
data = data.json()
File "C:\Users\RAM LLC\AppData\Local\Programs\Python\Python38-32\lib\site-packages\requests\models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "C:\PythonPrograms\Python\Python38-32\LIb\json_init
.py", line 357, in loads
return _default_decoder.decode(s)
File "C:\PythonPrograms\Python\Python38-32\LIb\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\PythonPrograms\Python\Python38-32\LIb\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

@uad1098
Copy link

uad1098 commented Mar 17, 2023

Solved my problem: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Thanks to GabrieleTap - uninstalled and installed yfinance and pandas-datareader

GabrieleTap commented on Jul 19, 2021
This worked for me:

pip uninstall yfinance
pip uninstall pandas-datareader
pip install yfinance --upgrade --no-cache-dir
pip install pandas-datareader

@heetsamber
Copy link

from pandas_datareader import data as web
from datetime import datetime
import yfinance as yf

yf.pdr_override()

symbol = "000660.KS"

start = datetime(2016,3,1)
end = datetime(2016, 3, 31)
sk = web.get_data_yahoo(tickers=symbol, start=start,end=end) # yf.download()
print(sk)

@sanjayram15
Copy link

File "/Users/yeison/miniforge3/envs/tf-metal-0.6.0/lib/python3.10/site-packages/pandas_datareader/yahoo/daily.py", line 153, in _read_one_data data = j["context"]["dispatcher"]["stores"]["HistoricalPriceStore"] TypeError: string indices must be integers

if you find an solution kindly update an solution @ sanjayram1515

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.