pickle error when serialize cython class with lambda function

[kibamin]

I use pickle and dill for follow lambda function and work fine :

import dill
import pickle

f = lambda x,y: x+y
s = pickle.dumps(f)

or even when used in class, for example:

file

foo.py

class Foo(object):
    def __init__(self):
        self.f = lambda x, y: x+y 

file

test.py

import dill
import pickle
from foo import Foo

f = Foo()
s = pickle.dumps(f)  # or  s = dill.dumps(f)

but when build same file with format .pyx (foo.pyx) using cython, can't serialize with dill, pickle or cpickle, get this error :

Traceback (most recent call last):
File "/home/amin/anaconda2/envs/rllab2/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2878, in run_cod
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 1, in
a = pickle.dumps(c)
File "/home/amin/anaconda2/envs/rllab2/lib/python2.7/pickle.py", line 1380, in dumps
Pickler(file, protocol).dump(obj)
File "/home/amin/anaconda2/envs/rllab2/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/home/amin/anaconda2/envs/rllab2/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "/home/amin/anaconda2/envs/rllab2/lib/python2.7/pickle.py", line 425, in save_reduce
save(state)
File "/home/amin/anaconda2/envs/rllab2/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/home/amin/anaconda2/envs/rllab2/lib/python2.7/site-packages/dill/_dill.py", line 912, in save_module_dict
StockPickler.save_dict(pickler, obj)
File "/home/amin/anaconda2/envs/rllab2/lib/python2.7/pickle.py", line 655, in save_dict
self._batch_setitems(obj.iteritems())
File "/home/amin/anaconda2/envs/rllab2/lib/python2.7/pickle.py", line 669, in _batch_setitems
save(v)
File "/home/amin/anaconda2/envs/rllab2/lib/python2.7/pickle.py", line 317, in save
self.save_global(obj, rv)
File "/home/amin/anaconda2/envs/rllab2/lib/python2.7/pickle.py", line 754, in save_global
(obj, module, name))
PicklingError: Can't pickle <cyfunction Foo.init.. at 0x7f9ab1ff07d0>: it's not found as foo.lambda

setup.py file for build cython

setup.py

from distutils.core import setup
from Cython.Build import cythonize

setup(ext_modules=cythonize("foo.pyx"))

then run in terminal:

python setup.py build_ext --inplace

Any idea how I can fix this?

Thanks.

[mmckerns]

As I posted on SO -- I believe there are (currently) no known serializers that can pickle cython lambdas, or the vast majority of cython code. Indeed, it is much more difficult for python serializers to be able to pickle objects with C-extensions unless the authors of the C-extension code specifically build serialization instructions (as did numpy and pandas). In that vein... instead of a lambda, you could build a class with a __call__ method, so it acts like a function... and then add one or more of the pickle methods (__reduce__, __getstate__, __setstate__, or something similar)... and then you should be able to pickle instances of your class. It's a bit of work, but since this path has been used to pickle classes written in C++ -- I believe you should be able to get it to work for cython-built classes.

[mmckerns]

Also see #319 and #208 for reference.