Want to further removing noise #316
Comments
|
For fMRI, you can also remove global respiratory artifact using temporal ICA. For unstructured noise, we recommend parcellation. We are working on other methods for non-parcellated analyses, but have not yet published detailed guidelines on that. |
Hello, I have a question the CPU of my personal computer is 20 cores and 28 threads. During batch processing, I found that my CPU utilization is also relatively low, and the time is also relatively long, especially the "recon-all" step. May I ask how to efficiently use the CPU core and shorten the time of data preprocessing? |
|
I think the next FreeSurfer will help with this. The question will be whether it also fixes the issues with surface placement that were in FS 7. |
|
If you have multiple sessions to process, I suggest that you just run multiple sessions simultaneously – 1 core per session.
Cheers,
-MH
…--
Michael Harms, Ph.D.
-----------------------------------------------------------
Professor of Psychiatry
Washington University School of Medicine
Department of Psychiatry, Box 8134
660 South Euclid Ave. Tel: 314-747-6173
St. Louis, MO 63110 Email: ***@***.***
From: FHZMZDL ***@***.***>
Reply-To: Washington-University/HCPpipelines ***@***.***>
Date: Tuesday, November 19, 2024 at 8:00 AM
To: Washington-University/HCPpipelines ***@***.***>
Cc: Subscribed ***@***.***>
Subject: Re: [Washington-University/HCPpipelines] Want to further removing noise (Issue #316)
* External Email - Caution *
For fMRI, you can also remove global respiratory artifact using temporal ICA. For unstructured noise, we recommend parcellation. We are working on other methods for non-parcellated analyses, but have not yet published detailed guidelines on that.
Hello, I have a question the CPU of my personal computer is 20 cores and 28 threads. During batch processing, I found that my CPU utilization is also relatively low, and the time is also relatively long, especially the "recon-all" step. May I ask how to efficiently use the CPU core and shorten the time of data preprocessing?
—
Reply to this email directly, view it on GitHub<#316 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AA5TABUTVTN7T4XXYMUOI3D2BM75RAVCNFSM6AAAAABR7WGL5GVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBVG44TQOJYG4>.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
________________________________
The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.
|
Hi, when I run the GenericfMRISurfaceProcessingPipeline steps, I found that the amount of time special long, eg more than 23 h, but also in the running, I use slrum scheduling software runs seven subjects, Before GenericfMRISurfaceProcessingPipeline previous steps to run a successful and running time is similar |
|
How are you calling the pipeline? That one should be fairly fast. |
This is a patient subject login cost long time: |
|
How long is the fMRI? |
The step of rfMRI_REST1_RL cost more than 4hours and the step of rfMRI_REST1_LR also cost more than 4hours. |
|
How many timepoints? |
415 |
|
That should be very fast. Are you having issues with thread contention? |
There should be no problem, when running the Genetic Volume step, the time is normal, I do not know why it takes so long to run the Genetic Surface step |
|
fMRISurface is nearly entirely multi-threaded. If you try to run multiple of them in parallel without some kind of thread control, it could cause a large amount of thread contention and slow things down a lot. |
I I'm not quite sure if my setup is correct, thanks for your reply！ |
Smoothing and resampling run for a long time, I guess wb_command lacks dependencies? |
|
The volume predilate amount could be reduced, as noted in the comment (its purpose is to reduce ringing effects from cubic resampling, but -cifti-resample uses the average value of the edge to fill the background (instead of 0), and has for a while, so this is less of a concern than with 1.4.0). I'm not sure if that would make it take 4 hours on 2mm data, though. What version of workbench are you using? |
|
A mock test (91k file resampled to its own space) takes 10 minutes on 4 physical cores with the current 10mm predilate on a 1000 timepoint 2mm file. The smoothing step shouldn't take as long. Are you using a multi-socket machine? Those have more overhead for thread synchronization when all cores are used, it is generally best to restrict each process to a single socket so that thread synchronization is fast. For a quick and dirty method, you can export |
Hi, My workbench version is ,2.0.1 -cifti-resample cost a long time: |
The simulation test (v 2.0.0) only on the workbench took very little time, and the longer it took seemed to be the Workbench version. |
|
The test I ran was on the development branch of workbench, which should be effectively the same as 2.0.1. It looked like you initially wrote "version 1.2.3" and then edited it to 2.0.1. Version 1.2.3 was released in 2016, and the speed of dilation was substantially improved in v1.4.2 (in 2020). Can you clarify if you are seeing this slowness with workbench v2.0.1? |
I used workbench"version "2.0.0" test to run one subject (without slrum scheduling software), as shown in 0029-2 (login1), the running time was relatively fast, about 12 minutes, and then I tested 8 subjects (slrum scheduling software). The time to run a subject (RL/LR) is relatively long, about 4-5h (login2), The process file ${ResultsFolder}/${NameOffMRI}_temp_template.dlabel.nii this step is very slow (SubcorticalProcessing.sh line119), As can be seen from login2, the running time is very slow, and the CPU monitoring software can also see that the process has been stuck in the process file ${ResultsFolder}/${NameOffMRI}_temp_template.dlabel.nii step. login1 run for patient 0029-2(RL): login1 run for patient 0029-2(LR): login2 run for patient by using slrum scheduling software(RL): login2 run for patient by using slrum scheduling software(LR): |
|
Try adding |
When I preprocessed the DWI data, I encountered an error that apparently meant numpy was missing from fsl, so I downloaded numpy 1.26.4. I still encountered the lack of numpy when running ICAFIX, but finally I was able to run successfully. My fsl version is 6.07.15, the default python is 3.12 and numpy1.24.4 seem incompatible, several downloads were unsuccessful, so I downloaded numpy1.26.4. numpy1.24.4 is recommended for HCPpipeline. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, After finishing the minimally preprocessed the series data for fMRI (denoising based on the ICA-fix algorithm) and dMRI, I want to further remove noise, How can I choose the way for denoising noise? Hope to recommend a more common way of noise reduction.
Hope your reply!
The text was updated successfully, but these errors were encountered: