Dynamic iterables

[dmwelch]

@satra @chrisfilo Having runtime defined iterable attributes would be awesome, especially for big data sets. Issue has been raised on the mailing list before: https://groups.google.com/forum/#!msg/nipy-user/pYW70-xAeu4/4MNbiDVQvAMJ

The current option is a MapNode of a Function that contains the workflow to iterate...

Cheers,
Dave

[satra]

@dmwelch - it would be nice right!

but we would need to fundamentally re-architect some things. i have a little bit of bandwidth in the next month and i'll see if i can come up with an architecture plan for this.

[belevtsoff]

+1, I would love this feature

[AlanKuurstra]

+1, This feature would be useful in combination with the experimental remove_node_directories for large datasets whose processing eats up disk space.

I have an fmri dataset that must be registered to a high resolution atlas. A number of preprocessing steps are done before the fmri file is split into separate time volumes. It wold be nice to send the split volumes into the next node's iterables list. After the split, each volume would be processed by a chain of subsequent nodes separately. The subsequent chain of node directories would be removed (freeing up space) before moving on to the next time volume.

[effigies]

This will not be implemented in Nipype 1, but is being implemented in Pydra, which will be the engine for Nipype 2.