JoinNode bug appears only on a different computer

[bpinsard]

Summary

I have a simple code that have an iterable connected to a join node.

    subjects_info = pe.Node(
        utility.IdentityInterface(fields=['sub']),
        run_without_submitting=True,
        name='subjects_info')
    subjects_info.iterables = [('sub', subject_ids)]


    template_args = dict(
        t1 = [['sub','ses','anat/sub-*_ses-SSV*_acq-MEMPRAGE_T1w.nii.gz']],
        t2 = [['sub','ses','anat/sub-*_ses-SSV*_T2w.nii.gz']],
        fmri = [['sub','ses','func/*.nii.gz']],
        fieldmap_mag=[['sub','ses','fmap/*phasediff.nii.gz']], # names are swapped !!                                         
        fieldmap_field=[['sub','ses','fmap/*magnitude1.nii.gz']],) # names are swapped !!                                     
    sessions_info = pe.Node(
        nio.DataGrabber(infields=['sub', 'ses'],
                        outfields=template_args.keys(),
			sort_filelist = True,
			raise_on_empty = False,
                        base_directory = mri_data_dir,
                        template=''),
	run_without_submitting=True,
        name='sessions_info')
    sessions_info.inputs.template = 'sub-%07d/ses-SSV%d/%s'
    sessions_info.inputs.template_args = template_args

    sessions_info.iterables = [('ses',range(1,15))]

    all_sessions = pe.JoinNode(
	utility.IdentityInterface(fields=template_args.keys()),
	joinsource = 'sessions_info',
        run_without_submitting=True,
        unique=False,
        name='all_sessions')
    w = pe.Workflow(name='moco_multiband')
    w.base_dir = proc_dir

    w.connect([
        (subjects_info, sessions_info,[('sub',)*2]),
        (sessions_info, all_sessions, [(k,k) for k in template_args.keys()])
    ])

I tried to install nipype (master branch) on a HPC, and use the same pipeline as before, but when I try to run I get the following error:

Actual behavior

AttributeError                            Traceback (most recent call last)
<ipython-input-5-b9a5896f20e5> in <module>()
----> 1 w.run()

/home/bpinsard/.conda/envs/prod/lib/python2.7/site-packages/nipype-0.13.0_gff36588.dev-py2.7.egg/nipype/pipeline/engine/workfl
ows.pyc in run(self, plugin, plugin_args, updatehash)
    566         logger.info('Workflow %s settings: %s', self.name, to_str(sorted(self.config)))
    567         self._set_needed_outputs(flatgraph)
--> 568         execgraph = generate_expanded_graph(deepcopy(flatgraph))
    569         for index, node in enumerate(execgraph.nodes()):
    570             node.config = merge_dict(deepcopy(self.config), node.config)

/home/bpinsard/.conda/envs/prod/lib/python2.7/site-packages/nipype-0.13.0_gff36588.dev-py2.7.egg/nipype/pipeline/engine/utils.
pyc in generate_expanded_graph(graph_in)
    786         graph_in = _merge_graphs(graph_in, subnodes,
    787                                  subgraph, inode._hierarchy + inode._id,
--> 788                                  iterables, iterable_prefix, inode.synchronize)
    789 
    790         # reconnect the join nodes

/home/bpinsard/.conda/envs/prod/lib/python2.7/site-packages/nipype-0.13.0_gff36588.dev-py2.7.egg/nipype/pipeline/engine/utils.
pyc in _merge_graphs(supergraph, nodes, subgraph, nodeid, iterables, prefix, synchronize)
    513     # Copy the iterable subgraphs
    514     for i, params in enumerate(iterable_params):
--> 515         Gc = deepcopy(subgraph)
    516         ids = [n._hierarchy + n._id for n in Gc.nodes()]
    517         nodeidx = ids.index(nodeid)

.
.
.
here goes all deepcopy machinery
.
.
.
.
/home/bpinsard/.conda/envs/prod/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
    172             copier = getattr(x, "__deepcopy__", None)
    173             if copier:
--> 174                 y = copier(memo)
    175             else:
    176                 reductor = dispatch_table.get(cls)

/home/bpinsard/.conda/envs/prod/lib/python2.7/site-packages/nipype-0.13.0_gff36588.dev-py2.7.egg/nipype/interfaces/base.pyc in __deepcopy__(self, memo)
    645         # access all keys
    646         for key in self.copyable_trait_names():
--> 647             _ = getattr(self, key)
    648         # clone once
    649         dup = self.clone_traits(memo=memo)

AttributeError: 'DynamicTraitedSpec' object has no attribute 'fieldmap_magJ14'

So it seems that all iterate fields are listed in copyable_trait_names while it is not on my desktop with the exact same code, this is crazy. Any idea ?

Expected behavior

works perfectly on my desktop.

How to replicate the behavior

I don't know, this seems platform specific.

Platform details:

{'nibabel_version': '2.1.0', 'networkx_version': '1.11', 'numpy_version': '1.11.2', 'sys_platform': 'linux2', 'sys_version': '2.7.12 |Continuum Analytics, Inc.| (default, Jul 2 2016, 17:42:40) \n[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]', 'commit_source': u'installation', 'commit_hash': u'ff36588', 'pkg_path': '/home/bpinsard/.conda/envs/prod/lib/python2.7/site-packages/nipype-0.13.0_gff36588.dev-py2.7.egg/nipype', 'sys_executable': '/home/bpinsard/.conda/envs/prod/bin/python', 'traits_version': '4.6.0', 'scipy_version': '0.18.1'}

[satra]

@bpinsard - since we switched to pure setuptools it looks like nipype may get confused with a few things. to make sure we rule out such issues, could you please uninstall nipype (pip uninstall nipype a few times till it doesn't find any) and make sure that this location does not contain any nipype directories /home/bpinsard/.conda/envs/prod/lib/python2.7/site-packages/ .

also please remove any nipype* files from /home/bpinsard/.conda/envs/prod/

[bpinsard]

Cleaned everything, re-clone the git folder, the same problem occurs.

[bpinsard]

I checked with the pipy version, and also have the same bug.

[chrisgorgo]

Does using python 2 vs 3 make any difference?

…

On Fri, Dec 16, 2016 at 10:23 AM, bpinsard ***@***.***> wrote: I checked with the pipy version, and also have the same bug. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1752 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOkp2yg3ugkJ7mUkfToNJnt22TLbcn3ks5rItcIgaJpZM4LPOxr> .

[bpinsard]

Is nipype really python3 compatible? I am battling to have it work, and it crashes at deepcopy stuff again, but with a different message:

/home/bpinsard/.conda/envs/py3k/lib/python3.5/site-packages/nipype/pipeline/engine/workflows.py in _create_flat_graph(self)
    835         """Make a simple DAG where no node is a workflow."""
    836         logger.debug('Creating flat graph for workflow: %s', self.name)
--> 837         workflowcopy = deepcopy(self)
    838         workflowcopy._generate_flatgraph()
    839         return workflowcopy._graph


.
.
.


/home/bpinsard/.conda/envs/py3k/lib/python3.5/copy.py in deepcopy(x, memo, _nil)
    172                     reductor = getattr(x, "__reduce_ex__", None)
    173                     if reductor:
--> 174                         rv = reductor(4)
    175                     else:
    176                         reductor = getattr(x, "__reduce__", None)

TypeError: can't pickle dict_keys objects

[chrisgorgo]

We've been using nipype with python 3.5 on a daily basis now with no such problems. Is this master?

…

On Fri, Dec 16, 2016 at 1:03 PM, bpinsard ***@***.***> wrote: Is nipype really python3 compatible? I am battling to have it work, and it crashes at deepcopy stuff again, but with a different message: /home/bpinsard/.conda/envs/py3k/lib/python3.5/site-packages/nipype/pipeline/engine/workflows.py in _create_flat_graph(self) 835 """Make a simple DAG where no node is a workflow.""" 836 logger.debug('Creating flat graph for workflow: %s', self.name) --> 837 workflowcopy = deepcopy(self) 838 workflowcopy._generate_flatgraph() 839 return workflowcopy._graph . . . /home/bpinsard/.conda/envs/py3k/lib/python3.5/copy.py in deepcopy(x, memo, _nil) 172 reductor = getattr(x, "__reduce_ex__", None) 173 if reductor: --> 174 rv = reductor(4) 175 else: 176 reductor = getattr(x, "__reduce__", None) TypeError: can't pickle dict_keys objects — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1752 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOkp3qlzLCgMWjvj0TniViJR8X_NvK3ks5rIvyjgaJpZM4LPOxr> .

[bpinsard]

this is both with pip and github master.
Are you using anaconda?

[oesteban]

I daily use anaconda + python 2.7 without issues in a HPC. Could you print out the environment in your laptop? Maybe looking at the differences we can get some lead to the problem.

[bpinsard]

I just upgraded my traits using pip on my desktop and the same bug as on the HPC occurs.
with traits4.5 it's ok, with 4.6 it bugs, can you try on yours?

[mgxd]

@bpinsard could you make a simple standalone working example of the error? If this is a bug, it would be great to have a test

[bpinsard]

Ok, here is the example, just to know it requires iterable and joinnode nested in a iterable (eg. iterate sessions then join in a subject iterable) to happen.

import nipype.pipeline.engine as pe
import nipype.interfaces.utility as iu

n_iter1 = pe.Node(iu.IdentityInterface(fields=['arg1']),name='node_iter1')
n_iter1.iterables = [('arg1',range(10))]

n_iter2 = pe.Node(iu.IdentityInterface(fields=['arg1','arg2']),name='node_iter2')
n_iter2.iterables = [('arg2',range(10))]

n_join = pe.JoinNode(iu.IdentityInterface(fields=['arg2']),joinsource = 'node_iter2',name='node_join')

w = pe.Workflow(name='iter_join')
w.connect([
	(n_iter1, n_iter2, [('arg1',)*2]),
	(n_iter2, n_join, [('arg2',)*2])
])
w.base_dir ='/tmp'
w.run()

with traits4.5 it works
with traits4.6 it doesn't

[mgxd]

Thanks - seems to be happening with any iterable workflow. Will have to investigate.

EDIT: Updating to master solved single iterable, but still problems with dual iterable expansions