PyDesigner terminated with partial output - did it run out of memory? #318
Description
Describe the Issue
I ran pydesigner (resuming from a partial run), and it terminated (without any messages) at the following stage:
$ docker run -it --rm -v /home/avanti.shrikumar/repos/av_pku_analysis/prototyping/STI60510710_non_compliant/niftis:/input -v /home/avanti.shrikumar/repos/av_pku_analysis/prototyping/STI60510710_non_compliant/pydesigner_out:/output dmri/neurodock pydesigner --verbose --resume --standard --nthreads 2 --output /output /input/dmri_main/5d434276-0b88-4a8b-a5a8-086a7269d1ba_conv.nii.gz /input/dmri_flipped/1ec06ae4-b0fa-4a57-bac6-0af699ff4f64_conv.nii.gz
Insufficient information in .json file to determine Fourier status. Assuming DWI is partial Fourier
[WARNING] Given DWI is partial fourier, overriding --degibbs; no unringing correction will be done to avoid artifacts.Use the "--adv" flag to run forced corrections.
Using 2 threads.
Plotting motion...
dwiextract: [100%] extracting volumes
mrmath: [100%] preloading data for "/output/B0_ALL.mif"
mrmath: [100%] computing mean along axis 3...
mrcalc: [100%] computing: (finite (/output/B0_MEAN.mif) ? /output/B0_MEAN.mif : 0)
dwiextract: [100%] extracting volumes
mrmath: [100%] preloading data for "/output/b1000_ALL.mif"
mrmath: [100%] computing mean along axis 3...
mrcalc: [100%] computing: (finite (/output/b1000_MEAN.mif) ? /output/b1000_MEAN.mif : 0)
mrcalc: [100%] computing: ((log ((/output/S0.mif / /output/S1000.mif)) / 1) > 2)
dwiextract: [100%] extracting volumes
mrmath: [100%] preloading data for "/output/B0_ALL.mif"
mrmath: [100%] computing mean along axis 3...
mrcalc: [100%] computing: (finite (/output/B0_MEAN.mif) ? /output/B0_MEAN.mif : 0)
mrcalc: [WARNING] existing output files will be overwritten
mrcalc: [100%] computing: (/output/brain_mask.nii * /output/csf_mask.nii)
Computing SNR: dwi_raw.nii
* B0...
* B1000...
* B3000...
Computing SNR: dwi_preprocessed.nii
* B0...
* B1000...
* B3000...
Image dwi_preprocessed.nii loaded successfully
Processing with 2 workers...
Protocol(s) detected: DTI, DKI
IRLLS Noise Estimation: [100%]
IRLLS Outlier Detection: [100%]
/output/metrics_qc/fitting/outliers_irlls.nii
Plotting outliers...
Constrained Tensor Fit: [0%]/usr/local/lib/python3.7/dist-packages/cvxpy/problems/problem.py:1386: UserWarning: Solution may be inaccurate. Try another solver, adjusting the solver settings, or solve with verbose=True for more information.
"Solution may be inaccurate. Try another solver, "
Constrained Tensor Fit: [1%]/usr/local/lib/python3.7/dist-packages/cvxpy/problems/problem.py:1386: UserWarning: Solution may be inaccurate. Try another solver, adjusting the solver settings, or solve with verbose=True for more information.
"Solution may be inaccurate. Try another solver, "
Constrained Tensor Fit: [100%]
DTI Parameters: [100%]
DTI ODF: [100%]
(as an aside, I had to delete the files csf_mask.nii, S0.mif and S1000.mif to get it to run, because otherwise it was throwing an error saying it refused to overrwrite those files).
The output files under "metrics" are as follows:
ti_ad.nii dti_fa.nii dti_fe.nii dti_md.nii dti_rd.nii dti_trace.nii DT.nii KT.nii
I understand that there are supposed to be more. I was doing this test on a very lightweight amazon workspace instance with only limited memory - is it possible that pydesigner ran out of memory and that's why it terminated? How much memory should I expect the steps to take?
Thanks,
Avanti