Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TST] Refactoring CircleCI setup #1092

Closed
wants to merge 13 commits into from

Conversation

oesteban
Copy link
Member

@oesteban oesteban commented May 1, 2018

Moves --use-aroma to ds054 with two ideas:

  • Shorten ds005 which was growing too large (CIFTI, partial exec, etc.)
  • Remove SyN from ds054 because it has fieldmaps, and make time for AROMA.

Overall, this PR should shorten the runtime of tests.

Also, figures/ folder has been pruned out of the outputs check (reports can be visualized and this check is a source of many errors).

When finished, this should fix #1090.

Moves `--use-aroma` to ds054 with two ideas:
  * Shorten ds005 which was growing too large (CIFTI, partial exec, etc.)
  * Remove SyN from ds054 because it has fieldmaps, and make time for AROMA.

Overall, this PR should shorten the runtime of tests.

Also, figures/ folder has been pruned out of the outputs check (reports can be visualized and this check is a source of many errors).

When finished, this should fix nipreps#1090.
@oesteban oesteban requested a review from effigies May 1, 2018 18:59
@oesteban
Copy link
Member Author

oesteban commented May 1, 2018

Something weird is happening. Moving AROMA to ds054 now gives:

Traceback (most recent call last):
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/plugins/multiproc.py", line 68, in run_node
    result['result'] = node.run(updatehash=updatehash)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 480, in run
    result = self._run_interface(execute=True)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 564, in _run_interface
    return self._run_command(execute)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/pipeline/engine/nodes.py", line 644, in _run_command
    result = self._interface.run(cwd=outdir)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py", line 520, in run
    runtime = self._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/interfaces/report_base.py", line 51, in _run_interface
    ReportCapableInterface, self)._run_interface(runtime)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py", line 1020, in _run_interface
    self.raise_exception(runtime)
  File "/usr/local/miniconda/lib/python3.6/site-packages/niworkflows/nipype/interfaces/base/core.py", line 957, in raise_exception
    ).format(**runtime.dictcopy()))
RuntimeError: Command:
ICA_AROMA.py -den nonaggr -i /scratch/fmriprep_wf/single_subject_100185_wf/func_preproc_task_machinegame_run_01_wf/ica_aroma_wf/smooth/vol0000_xform-00000_merged_smooth.nii.gz -meldir /scratch/fmriprep_wf/single_subject_100185_wf/func_preproc_task_machinegame_run_01_wf/ica_aroma_wf/melodic -mc /scratch/fmriprep_wf/single_subject_100185_wf/func_preproc_task_machinegame_run_01_wf/bold_hmc_wf/normalize_motion/motion_params.txt -o out
Standard output:

------------------------------- RUNNING ICA-AROMA ------------------------------- 
--------------- 'ICA-based Automatic Removal Of Motion Artifacts' --------------- 

Warning! Please check whether the determined TR (of 1.0s) is correct!

Step 1) MELODIC
  - The specified MELODIC directory does not contain the required files to run ICA-AROMA. MELODIC will be run seperately.
Standard error:



An exception has been thrown
Logic error:- detected by Newmat: index error: requested index = 2

MatrixType = Rect   # Rows = 1; # Cols = 1

Traceback (most recent call last):
  File "/opt/ICA-AROMA/ICA_AROMA.py", line 193, in <module>
    aromafunc.runICA(fslDir, inFile, outDir, melDir, mask, dim, TR)
  File "/opt/ICA-AROMA/ICA_AROMA_functions.py", line 94, in runICA
    nrICs = int(float(subprocess.getoutput(cmd)))
ValueError: could not convert string to float: 'ERROR: Could not open file'
Return code: 1

And the weird issue of work/ subfolders not being stored as artifacts is replicated.

@effigies
Copy link
Member

effigies commented May 1, 2018

This looks reasonable, but I guess we'll see when tests finish.

@@ -575,9 +577,10 @@ jobs:
- run:
name: Clean-up work directory
command: |
sudo find /tmp/ds000210/work -not -name "*.svg" -not -name "*.html" -not -name "*.rst" \
sudo chown $(id -un):$(id -gn) -R /tmp/ds000210
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, this solves the mystery of the cropped work/ directories. However, now we get:

Uploading /tmp/ds000210/derivatives/fmriprep/sub-02/figures/sub-02_task-cuedSGT_run-01_echo-1_bold_flirt.svg: SKIPPED, file is empty

My impression is that FMRIPREP has started to write empty files at some point o_o

self._results['compression'].append(_copy_any(fname, out_file))

if len(self._results['compression']) == 1:
self._results['compression'] = self._results['compression'][0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this is over here, now. So I'll just re-note that you can drop these two lines.

oesteban added a commit to oesteban/fmriprep that referenced this pull request May 2, 2018
Moves `--use-aroma` to ds054 with two ideas:
  * Shorten ds005 which was growing too large (CIFTI, partial exec, etc.)
  * Remove SyN from ds054 because it has fieldmaps, and make time for AROMA.

Overall, this PR should shorten the runtime of tests.

Also, figures/ folder has been pruned out of the outputs check (reports can be visualized and this check is a source of many errors).

Supersedes nipreps#1092 (close nipreps#1092)
@oesteban
Copy link
Member Author

oesteban commented May 2, 2018

Continued in #1098. Fix for #1090 will be addressed in a new PR.

@oesteban oesteban closed this May 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI: Tests failing on ds005
2 participants