Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API: allow read_pickle to read from strings (and not just files) #5924

Closed
jreback opened this issue Jan 13, 2014 · 7 comments
Closed

API: allow read_pickle to read from strings (and not just files) #5924

jreback opened this issue Jan 13, 2014 · 7 comments
Labels
Enhancement IO Pickle read_pickle, to_pickle

Comments

@jreback
Copy link
Contributor

jreback commented Jan 13, 2014

brought up in #5661 , #5894

@ghost
Copy link

ghost commented Jan 15, 2014

omg #5954

@jreback
Copy link
Contributor Author

jreback commented Jan 15, 2014

should we have pd.read_pickles/pd.read_jsons/pd.read_msgpacks ?

@ghost
Copy link

ghost commented Jan 15, 2014

Yeah, I thought of that. I think that'd be fine.

@jreback
Copy link
Contributor Author

jreback commented Jan 15, 2014

then could have read_csvs to be consistent, instead of constantly read_csv(StringIO(data)......)

should be straightforward to create these automatically in io/api.py even

@ghost
Copy link

ghost commented Jan 15, 2014

All sounds good to me.

@jreback jreback modified the milestones: 0.15.0, 0.14.0 Apr 9, 2014
@jreback jreback modified the milestones: 0.16.0, Next Major Release Mar 3, 2015
@datapythonista datapythonista modified the milestones: Contributions Welcome, Someday Jul 8, 2018
@jbrockmendel jbrockmendel added the IO Pickle read_pickle, to_pickle label Oct 22, 2019
@jakirkham
Copy link
Contributor

FWIW with pickle protocol 5 one can do the following...

In [1]: import pickle
   ...: 
   ...: import pandas as pd

In [2]: df = pd.DataFrame({"i": [1, 2, 3], "f": [0.0, 0.5, 1.0]})

In [3]: L = []
   ...: h = pickle.dumps(df, protocol=5, buffer_callback=L.append)

In [4]: pickle.loads(h, buffers=L)
Out[4]: 
   i    f
0  1  0.0
1  2  0.5
2  3  1.0

So it should be possible to feed in a series of buffers this way

@mroeschke mroeschke removed this from the Someday milestone Oct 13, 2022
@mroeschke
Copy link
Member

Closed related to #53767 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement IO Pickle read_pickle, to_pickle
Projects
None yet
5 participants