Can't pickle cyfunction Socket.set: it's not found as zmq.backend.cython.socket.set #208

Joshuaalbert · 2017-02-01T15:40:45Z

Hi again,

It seems I cannot pickle astropy objects. I can post a code that reproduces.

Josh

matsjoyce · 2017-02-01T16:14:40Z

Post it then. Code always helps.

Joshuaalbert · 2017-02-01T16:18:33Z

Looking for the time, I'm working at the moment on a deadline. Tonight or tomorrow

matsjoyce · 2017-02-01T16:22:21Z

Oh, well don't worry then. I thought you already had the code.

mmckerns · 2017-02-01T20:14:17Z

@Joshuaalbert: Also post the traceback you are seeing... and if you can, also post the version information of any packages you are using in your code. Please try to provide a minimal code that reproduces the error.

mmckerns · 2017-02-06T12:36:40Z

The phrasing of the error (see the title) suggests this is a duplicate of #56 (or related). If it's not a duplicate, I'm pretty sure that the 3rd party object you are after is not something that dill will specifically support. However, if you post code (and the associated traceback) that shows this is a more general problem, then maybe a solution will be implemented.

Joshuaalbert · 2017-02-06T13:22:12Z

Here you go, I reduced code down to the key part causing the error. It's to do with builtins. The stacktrace should be easily reproduced by running this.

https://gist.github.com/Joshuaalbert/af44cb3628719d4790f775b7a33bb377

The stacktrace is long but easily reproduced by running that script. Note the error comes when I try to store the RadioArray object which contains a Logger object. The issue is with the Logger object, I think because it stores a builtin method?

matsjoyce · 2017-02-06T13:35:21Z

Which version of python are you using? Works fine for me with both 3.6 and 2.7.

Joshuaalbert · 2017-02-06T13:42:11Z

python 2.7.12, dill 0.2.5

matsjoyce · 2017-02-06T13:43:57Z

OK, could you try the git version of dill, and see if it fixes this. Also, which OS are you using?

Joshuaalbert · 2017-02-06T13:59:08Z

I'm using Anaconda 3.0 to manage my environment on a windows 10 OS. I'll check a newer dill

Joshuaalbert · 2017-02-06T14:00:04Z

In the meantime here is the stacktrace:

---------------------------------------------------------------------------
PicklingError                             Traceback (most recent call last)
<ipython-input-1-9e5acfe8dcdf> in <module>()
     53 
     54 if __name__ == '__main__':
---> 55     PrepareData()
     56 
     57 

<ipython-input-1-9e5acfe8dcdf> in PrepareData()
     48 
     49     f = open(dataFile,'wb')
---> 50     dill.dump(dataDict,f)
     51     f.close()
     52     return

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\site-packages\dill\dill.pyc in dump(obj, file, protocol, byref, fmode, recurse)
    234             return
    235     # end hack
--> 236     pik.dump(obj)
    237     stack.clear()  # clear record of 'recursion-sensitive' pickled objects
    238     return

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in dump(self, obj)
    222         if self.proto >= 2:
    223             self.write(PROTO + chr(self.proto))
--> 224         self.save(obj)
    225         self.write(STOP)
    226 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    284         f = self.dispatch.get(t)
    285         if f:
--> 286             f(self, obj) # Call unbound method with explicit self
    287             return
    288 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\site-packages\dill\dill.pyc in save_module_dict(pickler, obj)
    833             # we only care about session the first pass thru
    834             pickler._session = False
--> 835         StockPickler.save_dict(pickler, obj)
    836         log.info("# D2")
    837     return

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_dict(self, obj)
    653 
    654         self.memoize(obj)
--> 655         self._batch_setitems(obj.iteritems())
    656 
    657     dispatch[DictionaryType] = save_dict

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in _batch_setitems(self, items)
    690                 k, v = tmp[0]
    691                 save(k)
--> 692                 save(v)
    693                 write(SETITEM)
    694             # else tmp is empty, and we're done

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    329 
    330         # Save the reduce() output and finally memoize the object
--> 331         self.save_reduce(obj=obj, *rv)
    332 
    333     def persistent_id(self, obj):

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
    423 
    424         if state is not None:
--> 425             save(state)
    426             write(BUILD)
    427 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    284         f = self.dispatch.get(t)
    285         if f:
--> 286             f(self, obj) # Call unbound method with explicit self
    287             return
    288 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\site-packages\dill\dill.pyc in save_module_dict(pickler, obj)
    833             # we only care about session the first pass thru
    834             pickler._session = False
--> 835         StockPickler.save_dict(pickler, obj)
    836         log.info("# D2")
    837     return

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_dict(self, obj)
    653 
    654         self.memoize(obj)
--> 655         self._batch_setitems(obj.iteritems())
    656 
    657     dispatch[DictionaryType] = save_dict

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in _batch_setitems(self, items)
    690                 k, v = tmp[0]
    691                 save(k)
--> 692                 save(v)
    693                 write(SETITEM)
    694             # else tmp is empty, and we're done

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    284         f = self.dispatch.get(t)
    285         if f:
--> 286             f(self, obj) # Call unbound method with explicit self
    287             return
    288 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\site-packages\dill\dill.pyc in save_instancemethod0(pickler, obj)
   1002     else:
   1003         pickler.save_reduce(MethodType, (obj.im_func, obj.im_self,
-> 1004                                          obj.im_class), obj=obj)
   1005     log.info("# Me")
   1006     return

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
    399         else:
    400             save(func)
--> 401             save(args)
    402             write(REDUCE)
    403 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    284         f = self.dispatch.get(t)
    285         if f:
--> 286             f(self, obj) # Call unbound method with explicit self
    287             return
    288 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_tuple(self, obj)
    552         if n <= 3 and proto >= 2:
    553             for element in obj:
--> 554                 save(element)
    555             # Subtle.  Same as in the big comment below.
    556             if id(obj) in memo:

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    329 
    330         # Save the reduce() output and finally memoize the object
--> 331         self.save_reduce(obj=obj, *rv)
    332 
    333     def persistent_id(self, obj):

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
    423 
    424         if state is not None:
--> 425             save(state)
    426             write(BUILD)
    427 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    284         f = self.dispatch.get(t)
    285         if f:
--> 286             f(self, obj) # Call unbound method with explicit self
    287             return
    288 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\site-packages\dill\dill.pyc in save_module_dict(pickler, obj)
    833             # we only care about session the first pass thru
    834             pickler._session = False
--> 835         StockPickler.save_dict(pickler, obj)
    836         log.info("# D2")
    837     return

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_dict(self, obj)
    653 
    654         self.memoize(obj)
--> 655         self._batch_setitems(obj.iteritems())
    656 
    657     dispatch[DictionaryType] = save_dict

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in _batch_setitems(self, items)
    690                 k, v = tmp[0]
    691                 save(k)
--> 692                 save(v)
    693                 write(SETITEM)
    694             # else tmp is empty, and we're done

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    284         f = self.dispatch.get(t)
    285         if f:
--> 286             f(self, obj) # Call unbound method with explicit self
    287             return
    288 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_list(self, obj)
    604 
    605         self.memoize(obj)
--> 606         self._batch_appends(iter(obj))
    607 
    608     dispatch[ListType] = save_list

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in _batch_appends(self, items)
    640                 write(APPENDS)
    641             elif n:
--> 642                 save(tmp[0])
    643                 write(APPEND)
    644             # else tmp is empty, and we're done

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    329 
    330         # Save the reduce() output and finally memoize the object
--> 331         self.save_reduce(obj=obj, *rv)
    332 
    333     def persistent_id(self, obj):

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
    423 
    424         if state is not None:
--> 425             save(state)
    426             write(BUILD)
    427 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    284         f = self.dispatch.get(t)
    285         if f:
--> 286             f(self, obj) # Call unbound method with explicit self
    287             return
    288 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\site-packages\dill\dill.pyc in save_module_dict(pickler, obj)
    833             # we only care about session the first pass thru
    834             pickler._session = False
--> 835         StockPickler.save_dict(pickler, obj)
    836         log.info("# D2")
    837     return

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_dict(self, obj)
    653 
    654         self.memoize(obj)
--> 655         self._batch_setitems(obj.iteritems())
    656 
    657     dispatch[DictionaryType] = save_dict

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in _batch_setitems(self, items)
    685                 for k, v in tmp:
    686                     save(k)
--> 687                     save(v)
    688                 write(SETITEMS)
    689             elif n:

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    329 
    330         # Save the reduce() output and finally memoize the object
--> 331         self.save_reduce(obj=obj, *rv)
    332 
    333     def persistent_id(self, obj):

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
    423 
    424         if state is not None:
--> 425             save(state)
    426             write(BUILD)
    427 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    284         f = self.dispatch.get(t)
    285         if f:
--> 286             f(self, obj) # Call unbound method with explicit self
    287             return
    288 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\site-packages\dill\dill.pyc in save_module_dict(pickler, obj)
    833             # we only care about session the first pass thru
    834             pickler._session = False
--> 835         StockPickler.save_dict(pickler, obj)
    836         log.info("# D2")
    837     return

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_dict(self, obj)
    653 
    654         self.memoize(obj)
--> 655         self._batch_setitems(obj.iteritems())
    656 
    657     dispatch[DictionaryType] = save_dict

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in _batch_setitems(self, items)
    685                 for k, v in tmp:
    686                     save(k)
--> 687                     save(v)
    688                 write(SETITEMS)
    689             elif n:

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    329 
    330         # Save the reduce() output and finally memoize the object
--> 331         self.save_reduce(obj=obj, *rv)
    332 
    333     def persistent_id(self, obj):

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
    423 
    424         if state is not None:
--> 425             save(state)
    426             write(BUILD)
    427 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    284         f = self.dispatch.get(t)
    285         if f:
--> 286             f(self, obj) # Call unbound method with explicit self
    287             return
    288 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\site-packages\dill\dill.pyc in save_module_dict(pickler, obj)
    833             # we only care about session the first pass thru
    834             pickler._session = False
--> 835         StockPickler.save_dict(pickler, obj)
    836         log.info("# D2")
    837     return

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_dict(self, obj)
    653 
    654         self.memoize(obj)
--> 655         self._batch_setitems(obj.iteritems())
    656 
    657     dispatch[DictionaryType] = save_dict

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in _batch_setitems(self, items)
    685                 for k, v in tmp:
    686                     save(k)
--> 687                     save(v)
    688                 write(SETITEMS)
    689             elif n:

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    284         f = self.dispatch.get(t)
    285         if f:
--> 286             f(self, obj) # Call unbound method with explicit self
    287             return
    288 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\site-packages\dill\dill.pyc in save_instancemethod0(pickler, obj)
   1002     else:
   1003         pickler.save_reduce(MethodType, (obj.im_func, obj.im_self,
-> 1004                                          obj.im_class), obj=obj)
   1005     log.info("# Me")
   1006     return

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_reduce(self, func, args, state, listitems, dictitems, obj)
    399         else:
    400             save(func)
--> 401             save(args)
    402             write(REDUCE)
    403 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    284         f = self.dispatch.get(t)
    285         if f:
--> 286             f(self, obj) # Call unbound method with explicit self
    287             return
    288 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_tuple(self, obj)
    552         if n <= 3 and proto >= 2:
    553             for element in obj:
--> 554                 save(element)
    555             # Subtle.  Same as in the big comment below.
    556             if id(obj) in memo:

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save(self, obj)
    315         # Check for string returned by reduce(), meaning "save as global"
    316         if type(rv) is StringType:
--> 317             self.save_global(obj, rv)
    318             return
    319 

C:\Users\josh\Anaconda3\envs\mayavi_env\lib\pickle.pyc in save_global(self, obj, name, pack)
    752             raise PicklingError(
    753                 "Can't pickle %r: it's not found as %s.%s" %
--> 754                 (obj, module, name))
    755         else:
    756             if klass is not obj:

PicklingError: Can't pickle <cyfunction Socket.set at 0x0000000003E88D38>: it's not found as zmq.backend.cython.socket.set

pseudotensor · 2019-05-24T08:27:26Z

Is this every going to be fixed?

mmckerns · 2019-06-26T18:08:55Z

Upon further inspection, I believe the issue is at root the same as #56. I'm closing it as duplicate.

mmckerns mentioned this issue Dec 23, 2018

AttributeError: module 'h2oaicore.models' has no attribute 'arithmetic_mean' #297

Closed

mmckerns added the duplicate label Jun 26, 2019

mmckerns closed this as completed Jun 26, 2019

mmckerns added this to the dill-0.3.0 milestone Jun 26, 2019

mmckerns mentioned this issue Apr 13, 2020

pickle error when serialize cython class with lambda function #360

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't pickle cyfunction Socket.set: it's not found as zmq.backend.cython.socket.set #208

Can't pickle cyfunction Socket.set: it's not found as zmq.backend.cython.socket.set #208

Joshuaalbert commented Feb 1, 2017

matsjoyce commented Feb 1, 2017

Joshuaalbert commented Feb 1, 2017 •

edited

Loading

matsjoyce commented Feb 1, 2017

mmckerns commented Feb 1, 2017

mmckerns commented Feb 6, 2017

Joshuaalbert commented Feb 6, 2017 •

edited

Loading

matsjoyce commented Feb 6, 2017

Joshuaalbert commented Feb 6, 2017

matsjoyce commented Feb 6, 2017

Joshuaalbert commented Feb 6, 2017

Joshuaalbert commented Feb 6, 2017

pseudotensor commented May 24, 2019

mmckerns commented Jun 26, 2019

Can't pickle cyfunction Socket.set: it's not found as zmq.backend.cython.socket.set #208

Can't pickle cyfunction Socket.set: it's not found as zmq.backend.cython.socket.set #208

Comments

Joshuaalbert commented Feb 1, 2017

matsjoyce commented Feb 1, 2017

Joshuaalbert commented Feb 1, 2017 • edited Loading

matsjoyce commented Feb 1, 2017

mmckerns commented Feb 1, 2017

mmckerns commented Feb 6, 2017

Joshuaalbert commented Feb 6, 2017 • edited Loading

matsjoyce commented Feb 6, 2017

Joshuaalbert commented Feb 6, 2017

matsjoyce commented Feb 6, 2017

Joshuaalbert commented Feb 6, 2017

Joshuaalbert commented Feb 6, 2017

pseudotensor commented May 24, 2019

mmckerns commented Jun 26, 2019

Joshuaalbert commented Feb 1, 2017 •

edited

Loading

Joshuaalbert commented Feb 6, 2017 •

edited

Loading