Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Control low-spatial-frequency and high-amplitude deformations for nonlinear transformateions #734

Closed
TheChymera opened this issue Mar 13, 2019 · 27 comments
Closed

Control low-spatial-frequency and high-amplitude deformations for nonlinear transformateions #734

TheChymera opened this issue Mar 13, 2019 · 27 comments

Comments

@TheChymera
Copy link
Contributor

I am having this issue (unsure if it is a bug, a missing feature, missing documentation, or inaccessibility of documentation) whereby the nonlinear transformation phase of an antsRegistration call ends up severely warping my image.

In most cases the transformation looks acceptable:

right

But in some it does not:

wrong

Interestingly this seems to only happen with specific moving images (I have been unable to determine the common denominator) and specific fixed images (e.g. the first but not the second one of our templates):

template

_template

I assume this has to do with the black border, but I was unable to test, because at this point the templates are pretty much “given”.

In my quest to resolve this I have manually iterated through countless parameter variations based on the instructions here, here and on incremental empirical evidence. It has boiled down to this:

antsRegistration --float --collapse-output-transforms 1 --dimensionality 3 --initial-moving-transform [ /usr/share/mouse-brain-atlases/ambmc_200micron.nii, /mnt/overflow/test_ambmc/test2/generic_work/_ind_type_4/s_biascorrect/sub-4004_ses-ofM_acq-TurboRARElowcov_T2w_corrected.nii.gz, 0 ] --initialize-transforms-per-stage 0 --interpolation Linear --output output_ --transform Rigid[ 0.1 ] --metric MeanSquares[ /usr/share/mouse-brain-atlases/ambmc_200micron.nii, /mnt/overflow/test_ambmc/test2/generic_work/_ind_type_4/s_biascorrect/sub-4004_ses-ofM_acq-TurboRARElowcov_T2w_corrected.nii.gz, 1, 64, Regular, 0.3 ] --convergence [ 1000x1000x1000, 1e-14, 30 ] --smoothing-sigmas 2.0x1.0x0.0vox --shrink-factors 4x2x1 --use-estimate-learning-rate-once 0 --use-histogram-matching 1 --masks [ /usr/share/mouse-brain-atlases/ambmc_200micron_mask.nii, NULL ] --transform Affine[ 0.1 ] --metric MI[ /usr/share/mouse-brain-atlases/ambmc_200micron.nii, /mnt/overflow/test_ambmc/test2/generic_work/_ind_type_4/s_biascorrect/sub-4004_ses-ofM_acq-TurboRARElowcov_T2w_corrected.nii.gz, 1, 64, Regular, 0.3 ] --convergence [ 200x100, 1e-08, 10 ] --smoothing-sigmas 1.0x0.0vox --shrink-factors 1x1 --use-estimate-learning-rate-once 0 --use-histogram-matching 1 --masks [ /usr/share/mouse-brain-atlases/ambmc_200micron_mask.nii, NULL ] --transform SyN[ 1e-08, 35.0, 0.0 ] --metric MI[ /usr/share/mouse-brain-atlases/ambmc_200micron.nii, /mnt/overflow/test_ambmc/test2/generic_work/_ind_type_4/s_biascorrect/sub-4004_ses-ofM_acq-TurboRARElowcov_T2w_corrected.nii.gz, 1, 16, Regular, 0.8
] --convergence [ 300, 1e-64, 10 ] --smoothing-sigmas 0.0vox --shrink-factors 1 --use-estimate-learning-rate-once 0 --use-histogram-matching 0 --masks [ /usr/share/mouse-brain-atlases/ambmc_200micron_mask.nii, NULL ] --winsorize-image-intensities [ 0.05, 0.95 ]
 --write-composite-transform 1

I use MI here to have the operation performed more rapidly. Adding CC+MI improves the results but does not sort out the issue completely. At any rate, this kind-of works, but not really. Please note that the issue is with the SyN phase alone, everything else works great.

More generally I have observed that:

  • histogram_matching=False is better
  • MI+CC is better than CC alone, which is better than MI alone

I can't really observe a reliable impact of editing the syn parameters, however. Past work indicates that since I work with mouse brains (having a much smaller scale than human brains, about 1:10 length units), I should reduce the first parameter by a factor of 10, but even reducing it by a factor of 10^8 seems to have done little to address the issue.
@gdevenyi
Copy link
Contributor

Suggestions

  • Fix your bias field with N4 first
  • Do a fully dense Rigid/Similarity/Affine before SyN with more blur levels
  • Really, use CC.
  • Don't winsorize your histogram

I have never had any issues with the default SyN parameters with Rats or Mice.

@stnava
Copy link
Member

stnava commented Mar 13, 2019 via email

@TheChymera
Copy link
Contributor Author

TheChymera commented Mar 13, 2019 •
edited
Loading

thank you for your speedy replies :)

@gdevenyi

  • the bias field is fixed, just that I apply the warp to the original image.
  • But I do do Rigid/Affine before, as for blur levels I came to notice that more of them meant more of the low-spatial-frequency deformations.
  • I know, but when I have to do manual machine learning it really helps to do it faster.
  • I have tried this out, but it sometimes ends up failing with:
WARNING: In /var/tmp/portage/sci-biology/ants-2.3.1/work/ants-2.3.1_build/ITKv5-install/include/ITK-5.0/itkObjectToObjectMetric.hxx, line 529
MeanSquaresImageToImageMetricv4 (0x56c04d0): No valid points were found during metric evaluation. For image metrics, verify that the images overlap appropriately. For instance, you can align the image centers by translation. For point-set metrics, verify that the fixed points, once transformed into the virtual domain space, actually lie within the virtual domain.

@stnava sorry for omitting that, please find a self-contained reproducible example here, there is an iteration step since the bad file doesn't always fail, but the good file always doesn't. Thanks for clarifying the unit for the SyN step sizes, that had been unclear to me. Sadly, increasing the totalFieldVarianceInVoxelSpace parameter does not seem to help as much as increasing updateFieldVarianceInVoxelSpace, and increasing both seems to nullify the slight positive effect acchieved by updateFieldVarianceInVoxelSpace. It is however entirely possible that my manual gradient descent has lead me into some local minimum which is entirely up the wrong tree. I would very much appreciate any suggestions from your side.

@gdevenyi
Copy link
Contributor

I think you meant to have a different link or an attachment that's missing. I don't see a runnable example with data.

@TheChymera
Copy link
Contributor Author

@gdevenyi oops, yes, that was an empty link, please check again now.

@gdevenyi
Copy link
Contributor

Okay, I see what's going on.

antsRegistration has some quirks with non-isotropic voxels. Whenever I'm solving a registration problem, I always upsample my fixed image to the highest res voxel in isotropic spacing.

If you do this:

ResampleImage 3 badatlas.nii resample.nii.gz 0.075x0.075x0.075 0 4

And use resample.nii.gz as your fixed image, all the regular antsRegistration suggestions work as expected.

@TheChymera
Copy link
Contributor Author

@gdevenyi interesting... but if you look at the headers, the images have the exact same level of anisotropy, so it really can't be just that, right?

Also, while completely removing the winsorization breaks the process, reducing the extent to which it is done has significantly improved the results, though I would again be very interested in further suggestions.

@gdevenyi
Copy link
Contributor

Another comment regarding choice of atlas, your atlas is "extracted" while your moving image isn't. This in general causes pain because you have edges in one image while there's no edge in the other.

@gdevenyi
Copy link
Contributor

Masking could fix this, but your mask is too generous so the edge is included in the registration.

@TheChymera
Copy link
Contributor Author

@gdevenyi yes, I have noticed that the main difference between the atlas that always works and the one which occasionally doesn't it the black rim. But I had thought the moving image would just be contracted into the non-black space (and this also happens e.g. with the “good” moving image I sent you). Any idea what causes this intance and how it might be combated without addding a resampling node to the preprocessing or recreating the template?

@TheChymera
Copy link
Contributor Author

@gdevenyi ok, so going back to your suggestion, I assume instead of

ResampleImage 3 badatlas.nii resample.nii.gz 0.075x0.075x0.075 0 4

you meant

ResampleImage 3 badmap.nii.gz _badmap.nii.gz 0.075x0.075x0.075 0 4

since the atlas is already isotropic. So I tried that and if anything it worsened the results, so I don't think it's the source of the problem or a good workaround.

@gdevenyi
Copy link
Contributor

Hrm, I didn't realize the atlas was isotropic.

So, I rephrase the problem. The atlas resolution is lower than some/all of the moving image dimensions. So the first thing ants does is throw away useful data since the fixed image determines the space the work is done. Upsampling the atlas fixes this.

@stnava
Copy link
Member

stnava commented Mar 13, 2019 via email

@ntustison
Copy link
Member

@gdevenyi -- I would just rephrase a couple things:

  • "antsRegistration has some quirks with non-isotropic voxels" --- I would suggest that the "quirks" associated with anisotropic sampling are general sampling issues as opposed to something particular to ANTs.

  • Same thing with "So the first thing ants does is throw away useful data". These are general implementation issues which are not specific to ANTs.

We would prefer that people reading this forum not come away thinking that ANTs is somehow uniquely deficient with respect to these issues.

@gdevenyi
Copy link
Contributor

gdevenyi commented Mar 14, 2019 •
edited
Loading

Absolutely agree @ntustison , much better phrasing.

@stnava
Copy link
Member

stnava commented Mar 14, 2019 via email

@TheChymera
Copy link
Contributor Author

TheChymera commented Mar 14, 2019 •
edited
Loading

The atlas resolution is lower than some/all of the moving image dimensions.

some, yes, the template is 0.2x0.2x0.2 whereas the structural scan is 0.167x0.075x0.65, the issue however, is that I am using the structural scan as an intermediary to register functional data (which is 0.312x0.281x0.65). Ideally I would not want to oversample the structural data too much due to both multiple comparison and memory concerns.

Still, I don't see how this would be the specific issue here, since over 50 other scans at the same resolution reliably register correctly, and it is only 2-3 where SyN introduces this strange roll/scale effect (which looks much more like some affine transformation).

Barring a change in the resolution of the target space, what else could I try? I basically just want to tell it to not make this kind of low spatial frequency high amplitude roll/scale. Mouse brains are much simpler than human brains, and I can reliably say that if the algorithm is moving 30% of the image more than 20 voxels away, its definitely wrong.

@stnava
Copy link
Member

stnava commented Mar 14, 2019 via email

@TheChymera
Copy link
Contributor Author

sorry, I didn't intend to overwhelm you with specific details about the data, I just wanted to clarify the details surrounding the topic of resolution which came up.

At any rate, the process at hand is very simple, I want to make slight (which I can also define in precise voxel/length units) transformations to an ellipsoid to make it fit into another. I assume this functionality is available in ANTs. The answers I have gotten were:

  • Reduce the extent to which the image is winsorized, which helped (thanks @gdevenyi) but did not fully fix the issue.
  • Enlarge totalFieldVarianceInVoxelSpace, which will not work with the example I have compiled for you.
  • Change the template/mask, which is infeasible.

Is there no ANTs parameter which reliably controls the amplitude of nonlinear transformations?

@stnava
Copy link
Member

stnava commented Mar 14, 2019 via email

@ntustison
Copy link
Member

Yes, as Brian mentioned, the total field variance constrains the deformation. It might not be the magic parameter which fixes your single example but that is the ANTs parameter in question. Another alternative to try is the BSplineSyN transform which, I find, tends to produce a more elastic- type solution.

@gdevenyi
Copy link
Contributor

While you're focusing on the SyN part of the registration it is my contention that this is a sign of a failure of the affine stage before it.

Here's my generic registration script for a affine+SyN registration, with its solutions to your good and bad images, using the upsampling. Without upsampling, the affine fails badly for this script, pointing to that being the source of your problems.

I suggest you QC your affine registrations as I do, by saving out an affine transformed image so you can see if your affine worked.

https://transfer.sh/ZcYku/regularization2.tar.xz

@gdevenyi
Copy link
Contributor

image

@stnava
Copy link
Member

stnava commented Mar 14, 2019 via email

@TheChymera
Copy link
Contributor Author

@gdevenyi thanks for giving this a shot!

Could you explain what I should be noticing in the image which you posted? It seems to me like all cases work out without showcasing my initial issue. I have also downloaded and looked at your archive, and I cannot find anything to suggest the following:

Without upsampling, the affine fails badly for this script

Looking at output/badmap-affine.nii.gz and output/goodmap-affine.nii.gz, both look equaly well fitted to the template:

Screenshot from 2019-03-14 15-28-50

On a tangential note: I have also mentioned above that the error looks more affine-like than SyN-like to me, but I had done QC, and figured out that up until the SyN phase good/bad images tended to be equally well registered.

@gdevenyi
Copy link
Contributor

I assumed that goodmap and badmap were two different input files.

I simply demonstrate that my script, using resample.nii.gz as a target both successfully register.

Here's the result if I use badatlas.nii.gz as the target.

image

@cookpa
Copy link
Member

cookpa commented Mar 14, 2019

The domain of the fixed image defines the domain of the virtual space the computation is performed. So changing the spacing of the fixed image changes the interpretation of several antsRegistration command line parameters including shrink factors, smoothing kernels, regularization parameters, and metric sampling.

It makes sense that resampling the atlas would change results, though it might be possible to arrive at the same results by adjusting other parameters.

I managed to get reasonable results with

antsRegistration --float 1 --dimensionality 3 \
 --initial-moving-transform [ badatlas.nii, badmap.nii.gz , 0 ] --transform Affine[ 0.1 ] \
 --metric MI[ badatlas.nii, badmap.nii.gz, 1, 32, Regular, 0.25] --convergence [ 100x50, 1E-8, 10 ] \
 --shrink-factors 2x1 --smoothing-sigmas 1x0vox --transform SyN[ 0.1, 3.0, 1 ] \
 --metric CC[ badatlas.nii, badmap.nii.gz, 1, 4 ] --convergence [ 25, 1E-8, 10 ] \
 --shrink-factors 1 --smoothing-sigmas 0vox \
 --verbose \
 --output [ badMapFullReg, badMapDeformed.nii.gz ]

As has been suggested above I did not use histogram matching or winsorization. I would be especially careful doing anything to the intensities if using mean squares metric, as the example script does. The other big variable, particularly if using different atlases, is going to be the initialization. If the geometric centers of the images align poorly, you might get a bad initial alignment.

@cookpa cookpa closed this as completed Aug 5, 2019
@chainhomelow chainhomelow mentioned this issue Aug 16, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@stnava @ntustison @cookpa @TheChymera @gdevenyi