Support for compiling FreeSurfer from source

[pwighton]

This PR adds support for compiling FreeSurfer from source.

A corresponding container for this PR is at pwighton/neurodocker:20220414

There are lots of moving pieces to this, and tons of edge cases. Feedback is highly appreciated!

Warning: by default, building from source includes a full checkout of FreeSurfer's git annex. This makes the resulting containers rather large (~5.58GB Gb). Thank goodness for neurodocker minify! :)

Use cases

Here are some use cases I've been working with. Hopefully more to come!

Compile the dev branch of FreeSurfer from source

This creates a ~5.58GB container:

docker run pwighton/neurodocker:20220414 generate docker \
    --base-image ubuntu:xenial \
    --pkg-manager apt \
    --yes \
    --freesurfer \
      method=source \
| docker build --no-cache --network host -t pwighton/fs-dev -	      

Compile an arbitraty fork and branch of Freesurfer

For example to compile the 20210813-gems branch of the https://github.com/pwighton/freesurfer.git fork:

docker run pwighton/neurodocker:20220414 generate docker \
    --base-image ubuntu:xenial \
    --pkg-manager apt \
    --yes \
    --freesurfer \
      method=source \
      repo=https://github.com/pwighton/freesurfer.git \
      branch=20210813-gems \
| docker build --no-cache --network host -t pwighton/fs-dev-20210813-gems -	      

Compile FreeSurfer from source; include dependencies needed to run the infant pipeline

This creates a ~41GB container:

docker run pwighton/neurodocker:20220414 generate docker \
    --base-image ubuntu:xenial \
    --pkg-manager apt \
    --yes \
    --niftyreg \
      version=master \
    --fsl \
      version=5.0.10 \
      method=binaries \
    --freesurfer \
      method=source \
      repo=https://github.com/freesurfer/freesurfer.git \
      branch=dev \
      infant_module=ON \
      dev_tools=ON \
      infant_model_s3=s3://freesurfer-annex/infant/model/dev/ \
      infant_model_s3_region=us-east-2 \
    --entrypoint '/bin/infant-container-entrypoint-aws.bash' \
| docker build --no-cache --network host -t pwighton/fs-infant-dev -

Compile FreeSurfer from source; include dependencies needed to run the petsurfer pipeline

This creates a ~42.6GB container

docker run pwighton/neurodocker:20220414 generate docker \
    --yes \
    --base-image ubuntu:xenial \
    --pkg-manager apt \
    --neurodebian \
      os_codename=xenial \
      version=usa-nh \
    --freesurfer \
      method=source \
      repo=https://github.com/freesurfer/freesurfer.git \
      branch=dev \
      license_base64=${FS_LICENSE_BASE64} \
      dev_tools=ON \
      minimal=OFF \
    --fsl \
      version=5.0.10 \
      method=binaries \
  | docker build --no-cache -t pwighton/petsurfer:dev -

Compile FreeSurfer from source; include support for training samseg alases

This creates a ~4.29GB container.

Since git annex isn't run, neither is make install, therefore the binary kvlBuildAtlasMesh is not in $PATH, but is located at /opt/freesurfer-dev/gems/bin/kvlBuildAtlasMesh

docker run pwighton/neurodocker:20220414 generate docker \
    --base-image ubuntu:xenial \
    --pkg-manager apt \
    --yes \
    --freesurfer \
      method=source \
      minimal=on \
      samseg_atlas_build=ON \
      git_annex_get=OFF \
      dev_tools=ON \
  | docker build --no-cache -t pwighton/samseg-atlas-build -

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!.

[kaczmarj]

@pwighton - thank you! we have been planning to have freesurfer from source for a while now... thanks for making this a reality. it will take me some time to review this PR, but i took a cursory look and looks good. thanks for including comments in the code.

[pwighton]

Thanks @kaczmarj! I'll also try to find people to review this.

[stebo85]

@pwighton - could you resolve the conflicts and update the pull request?

[pwighton]

Sure @stebo85, but I wont be able to look at this until September. Feel free to ping me again if I forget!

[pdlloyd]

Hi @pwighton, I'm working on building Freesurfer from source for a project I'm on to get it working with ARM64 on Linux. I'm following freesurfer/freesurfer#964 for a bit but I just found and read through this PR. I wish I'd found it weeks ago! In any case, I'd love to support this as well and help out where I can.

[pdlloyd]

I made a PR to pwighton:20220414-fs-source which will resolve this issue once @pwighton approves it. If he's not able to look at it for a while, I can also just duplicate this PR with my updated 20220414-fs-source branch and this PR can be closed. In cases like this, what would be considered best practice?

[satra]

@pdlloyd - you can wait a weekday or two (if not in a rush) for @pwighton to merge or open a separate PR incorporating all the commits (i.e send to neurodocker directly from branch that builds on pwighton's changes).

[pwighton]

Hi @pdlloyd, sorry for the lag and thanks very much for your help!

I'm a little swamped right now and wont have a chance to look at this for a while, but things were mostly working last year when I built the pwighton/neurodocker:20220414 container. It might be worth verifying the use cases described when I opened the PR are still working with the head of FreeSurfer's dev branch.

As for getting ARM64 working, it might make sense to create a separate neurodocker template that compiles itk from source. I've done that for ppc64 here and maybe that will work for AMR64 as well. Last I checked, that was the only dependency needed to run recon-all.

[pdlloyd]

@pwighton that's a good idea to create a separate ND template. I've spent some time working through compiling the dependency tree on Freesurfer's dev branch to get it working with ARM and I found some quirks that require some extra TLC.