Nifti file can not be opened after Jul 2020 commit

[issakomi]

This Nifti file can not be opened after ENH: Prefer to use sform over qform when both are set. The file was generated by VINCI.

The issue was posted in the Slicer forum.

In particular, this block scope, starting at the line 1913, prevents loading. The error message "ITK only supports orthonormal direction cosines. No orthonormal definition found!" is not applicable.

Looks like that IO expects a header like this (a kind of IndexToPointMatrix)

…
pixdim 0.0 0.865759 0.865759 0.796 0.0 0.0 0.0 0.0
…
srow_x   0.865759 0.0      0.0      -54.975697
srow_y   0.0      0.865759 0.0      -54.975697
srow_z   0.0      0.0      0.796    -37.411999

but there is

…
pixdim 0.0 0.865759 0.865759 0.796 0.0 0.0 0.0 0.0
…
srow_x    1.0   0.0    0.0     -54.975697
srow_y    0.0   1.0    0.0     -54.975697
srow_z    0.0   0.0    1.0     -37.411999

The file worked with earlier releases, orientation and spacing were correct:

Spacing: [0.865759, 0.865759, 0.796]
Origin: [54.9757, 54.9757, -37.412]
Direction:
-1  0  0
 0 -1  0
 0  0  1

The file can be opened If the block is removed. Spacing and orientation will be like above again, sform will be preferred.

Just FYI, please close the issue if not possible to fix.

[hjmjohnson]

@issakomi My reading is that the file is incorrectly written. Based on the documentation for the nifti1 file format standard, the pixel dims SHOULD NOT BE USED if the sform_code > 0.

That implies that the sform matrix should have the pixeldims embedded in the sform values.

Previous versions of ITK trusted that the pixel dims were correct. This version explicitly checks that the pixel-dimentions are properly encoded.

   METHOD 3 (used when sform_code > 0):
   -----------------------------------
   The (x,y,z) coordinates are given by a general affine transformation
   of the (i,j,k) indexes:

     x = srow_x[0] * i + srow_x[1] * j + srow_x[2] * k + srow_x[3]
     y = srow_y[0] * i + srow_y[1] * j + srow_y[2] * k + srow_y[3]
     z = srow_z[0] * i + srow_z[1] * j + srow_z[2] * k + srow_z[3]

   The srow_* vectors are in the NIFTI_1 header.  Note that no use is
   made of pixdim[] in this method.

[lassoan]

I've given up on Nifti many times and still not convinced that it has a place outside neuroimaging, but we keep getting complaints about it, so it is clear that people still try to use it in a wide range of applications. We should either strongly discourage everyone from using this file format or make it less painful to use it.

It seems that the root cause of most issues is that various Nifti reader implementations behave differently when encountering ambiguous/invalid input.

@hjmjohnson Would you consider changing the ITK implementation so that it behaves exactly the same way as other major Nifti readers? Alternatively, do you see a chance that you can convince other Nifti reader/writer developers to make their software work exactly the same way as ITK?

[gdevenyi]

@hjmjohnson Would you consider changing the ITK implementation so that it behaves exactly the same way as other major Nifti readers?

Perhaps a good place to start here is to document how widely the NIFTI "standard" is interpreted in implementations.

[hjmjohnson]

@lassoan I have been trying to make NIFTI work consistently. Unfortunately, other major Nifti readers do not behave consistently. Each medical imaging echo system seems to have different interpretations.

I have tried to make the NIFTI reader/writers maximize conformance with the ITK standards. I.E. make them as lossless as possible with the DICOM standard for reading/writing. The commonly used qform format is lossy with respect to the DICOM standard and has been found to introduce small inconsistencies in physical space representations.

Can you give an example of where the nifti reader in ITK performs differently than other important tools?

[lassoan]

Chris Rorden stated very recently when discussing this issue that NIfTI is simple and supported robustly by a huge number of tools, including major ones like SPM, FSL and AFNI. NIfTI was the lowest common denominator that allowed these tools to work robustly. The ITK implementation is the odd one out.

What I experienced was that that nibabel was able to load some images that ITK rejected (due to some axis orientation checks failed). Probably this is not the only problem but Chris Rorden referred to some more issues.

[hjmjohnson]

@lassoan Regarding axis orientation, sometimes that is due to round-off errors that other developers chose to ignore, and sometimes it is due to real-non-orthogonal space dimensions. The qform lossy orientation coding can make DICOM-->NIFTI-->DICOM conversions result in mis-alligned (by a small amount) data. I have run into this with many Prostate image mask generation data sets. The recent preference to store the sform was in response to that problem as a way to preserve more of the precision specified in the DICOM data physical space representation.

Many (most?) ITK algorithms do not work with non-orthogonal images and have chosen to reject them outright. Perhaps that decision was too conservative, but it was the consensus of the community at the time.

[hjmjohnson]

@lassoan I wish the rest of the world would adopt the NRRD data as the reference data. My feeling is that it is a much better thought-out data format. IMHO, NRRD also has some issues: 1) it's stuck on an SVN repository with too large of barriers to contributing to its development, and 2) most medical imaging eco-systems do not support it.

--- Perhaps we need a new data format standard :) built from the ground up. (I'm only 1/2 joking).

[gdevenyi]

--- Perhaps we need a new data format standard :) built from the ground up. (I'm only 1/2 joking).

Meanwhile MINC2.0 languishes in obscurity, solving both the NIFTI precision confusion and the BIDS metadata problem :-/

[lassoan]

The problem with MINC2 for me is that it depends on HDF5, which is a a huge and complex library. I particularly dislike HDF5 because when we had any ITK or Python build problems or crashes then many times it turned out to be related to HDF5. If bringing in a large and complex dependency is acceptable then I would rather go for DCMTK or GDCM and use DICOM.

NRRD is nice and simple, human-readable, almost self-descriptive. A reader or writer can be implemented in a few hours in any programming language. Therefore, it nicely complements DICOM.

[gdevenyi]

A reader or writer can be implemented in a few hours in any programming language.

Flashbacks to how NIFTI became the partially-implemented standard to begin with....

If this is true, someone should go write reference implementations for all the languages now to make sure it doesn't end up like NIFTI did.

[lassoan]

NRRD reader is already available in most commonly used languages in medical image computing (C++, Python, Matlab, R, Java, JavaScript, etc.). NRRD is a much easier file format to deal with compared to NIFTI because it does not have all the redundancies, ambiguities, and unnecessary complexities and limitations.

I really like the design goals and solutions implemented in MINC2. It is really just HDF5 that disqualifies it for even considering it for any of our projects. However, if you could port MINC features to OME-Zarr (use Zarr instead of HDF5 and harmonize existing OME-Zarr fields with MINC fields and add remaining MINC features to OME-Zarr) then it may be a winner file format. OME-Zarr is still an evolving file format, developed by an open, welcoming community, so it should not be hard to introduce any necessary changes/improvements.

[dzenanz]

This was fixed by #3339.

[issakomi]

This was fixed by #3339.

It is a little bit tricky, there are two similar topics in Slicer forum about Nifti and with the same ITK's error message. One was fixed by this commit. But the file in this discussion (link in the first post) can not be opened with current master. Тhe file was recognized as incorrect.

[gdevenyi]

@issakomi please open a new issue with details of the problem

[dzenanz]

https://github.com/InsightSoftwareConsortium/ITK/blob/v5.3rc04/Modules/IO/NIFTI/src/itkNiftiImageIO.cxx#L1903 does not only check whether sform is decomposable without skew, but also checks whether it represents the same spacing as qform.