Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Fix failing isAffine check due to insuffcient precision in calculations #3339

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 19 additions & 13 deletions Modules/IO/NIFTI/src/itkNiftiImageIO.cxx
Original file line number Diff line number Diff line change
Expand Up @@ -1753,10 +1753,15 @@ Normalize(std::vector<double> & x)
static bool
IsAffine(const mat44 & nifti_mat)
{
const vnl_matrix_fixed<float, 4, 4> mat(&(nifti_mat.m[0][0]));
vnl_matrix_fixed<double, 4, 4> mat;

for (int i = 0; i < 4; ++i)
for (int j = 0; j < 4; ++j)
mat[i][j] = double{ nifti_mat.m[i][j] };

// First make sure the bottom row meets the condition that it is (0, 0, 0, 1)
{
float bottom_row_error = itk::Math::abs(mat[3][3] - 1.0f);
double bottom_row_error = itk::Math::abs(mat[3][3] - 1.0);
Copy link
Member

@issakomi issakomi May 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is float epsilon several lines below (line 1769) intentional?

if (bottom_row_error > std::numeric_limits<float>::epsilon())

I thought that it might have been forgotten.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is float epsilon several lines below (line 1769) intentional?

@issakomi Good question. But I wonder, isn't std::numeric_limits<T>::epsilon() rather arbitrary anyway? I just don't know, but if mat[3][i] must be close to zero (for i = 0, 1, 2), then those matrix elements might as well be compared against std::numeric_limits<T>::min(), which if much smaller than std::numeric_limits<T>::epsilon().

Honestly I'm just wondering what is intended as well. I have no suggestion here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the bottom row should be precisely (0,0,0,1) - but I guess it's possible there might be some epsilon error when the inverse is calculated

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it was just forgotten.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there might be some epsilon error when the inverse is calculated

OK, but std::numeric_limits<T>::epsilon() isn't just some epsilon error. It is specifically the difference between 1 and the least value greater than 1 that is representable by floating point type T. https://www.cplusplus.com/reference/cfloat/ That's why I wonder if this specific epsilon should also be used for values that should be close to zero (rather than close to one). But I just don't know, honestly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One further idea here, currently the test is made with the sum as each entry added. Should we instead test the individual entries?

Changing to ::min(), or testing the individual entries will be substantially stricter than the existing implementation as well.

for (int i = 0; i < 3; ++i)
{
bottom_row_error += itk::Math::abs(mat[3][i]);
Expand All @@ -1767,28 +1772,29 @@ IsAffine(const mat44 & nifti_mat)
}
}

const float condition = vnl_matrix_inverse<float>(mat.as_matrix()).well_condition();
const double condition = vnl_matrix_inverse<double>(mat.as_matrix()).well_condition();
// Check matrix is invertible by testing condition number of inverse
if (!(condition > std::numeric_limits<float>::epsilon()))
if (!(condition > std::numeric_limits<double>::epsilon()))
{
return false;
}

// Calculate the inverse and separate the inverse translation component
// and the top 3x3 part of the inverse matrix
const vnl_matrix_fixed<float, 4, 4> inv4x4Matrix = vnl_matrix_inverse<float>(mat.as_matrix()).as_matrix();
const vnl_vector_fixed<float, 3> inv4x4Translation(inv4x4Matrix[0][3], inv4x4Matrix[1][3], inv4x4Matrix[2][3]);
const vnl_matrix_fixed<float, 3, 3> inv4x4Top3x3 = inv4x4Matrix.extract(3, 3, 0, 0);
const vnl_matrix_fixed<double, 4, 4> inv4x4Matrix = vnl_matrix_inverse<double>(mat.as_matrix()).as_matrix();
const vnl_vector_fixed<double, 3> inv4x4Translation(inv4x4Matrix[0][3], inv4x4Matrix[1][3], inv4x4Matrix[2][3]);
const vnl_matrix_fixed<double, 3, 3> inv4x4Top3x3 = inv4x4Matrix.extract(3, 3, 0, 0);

// Grab just the top 3x3 matrix
const vnl_matrix_fixed<float, 3, 3> top3x3Matrix = mat.extract(3, 3, 0, 0);
const vnl_matrix_fixed<float, 3, 3> invTop3x3Matrix = vnl_matrix_inverse<float>(top3x3Matrix.as_matrix()).as_matrix();
const vnl_vector_fixed<float, 3> inv3x3Translation = -(invTop3x3Matrix * mat.get_column(3).extract(3));
const vnl_matrix_fixed<double, 3, 3> top3x3Matrix = mat.extract(3, 3, 0, 0);
const vnl_matrix_fixed<double, 3, 3> invTop3x3Matrix =
vnl_matrix_inverse<double>(top3x3Matrix.as_matrix()).as_matrix();
const vnl_vector_fixed<double, 3> inv3x3Translation = -(invTop3x3Matrix * mat.get_column(3).extract(3));

// Make sure we adhere to the conditions of a 4x4 invertible affine transform matrix
const float diff_matrix_array_one_norm = (inv4x4Top3x3 - invTop3x3Matrix).array_one_norm();
const float diff_vector_translation_one_norm = (inv4x4Translation - inv3x3Translation).one_norm();
constexpr float normed_tolerance_matrix_close = 1e-2;
const double diff_matrix_array_one_norm = (inv4x4Top3x3 - invTop3x3Matrix).array_one_norm();
const double diff_vector_translation_one_norm = (inv4x4Translation - inv3x3Translation).one_norm();
constexpr double normed_tolerance_matrix_close = 1e-2;
return !((diff_matrix_array_one_norm > normed_tolerance_matrix_close) ||
(diff_vector_translation_one_norm > normed_tolerance_matrix_close));
}
Expand Down
5 changes: 5 additions & 0 deletions Modules/IO/NIFTI/test/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ itkNiftiImageIOTest9.cxx
itkNiftiImageIOTest10.cxx
itkNiftiImageIOTest11.cxx
itkNiftiImageIOTest12.cxx
itkNiftiImageIOTest13.cxx
itkNiftiReadAnalyzeTest.cxx
itkNiftiReadWriteDirectionTest.cxx
itkExtractSlice.cxx
Expand Down Expand Up @@ -96,3 +97,7 @@ itk_add_test(NAME itkNiftiReadWriteDirectionSmallVoxelTest
DATA{Input/SmallVoxels_nosform.nii.gz}
${ITK_TEST_OUTPUT_DIR}
)

itk_add_test(NAME itkNiftiSmallVoxelsAffinePrecisionTest
COMMAND ITKIONIFTITestDriver itkNiftiImageIOTest13
DATA{Input/SmallVoxels_AffinePrecision.nii.gz})
Comment on lines +101 to +103
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about using itkNiftiImageIOTest2 instead, like this:

itk_add_test(NAME itkNiftiSmallVoxelsAffinePrecisionTest
      COMMAND ITKIONIFTITestDriver itkNiftiImageIOTest2
              ${ITK_TEST_OUTPUT_DIR} itkNiftiIOShouldSucceed true DATA{Input/SmallVoxels_AffinePrecision.nii.gz})

Would that also work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That test with the original code does not fail.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That test with the original code does not fail.

Why not? Originally you added your SmallVoxels_AffinePrecision.nii.gz to an existing itk_add_test. My suggestion (above here) would be to do a new itk_add_test, referring to the old itkNiftiImageIOTest2 cxx file and your SmallVoxels_AffinePrecision.nii.gz file. But that would not work either?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't get why itkNiftiImageIOTest2 doesn't fail, because the test file SmallVoxels_AffinePrecision.nii.gz definitely causes an exception for me (running ANTs PrintHeader built against recent ITK).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverting the change, and running both tests.

$ ctest -R AffinePrecision --output-on-failure
Test project /home/gdevenyi/projects/itk-build
    Start 1157: itkNiftiSmallVoxelsAffinePrecisionTest
1/2 Test #1157: itkNiftiSmallVoxelsAffinePrecisionTest ............................***Failed    0.04 sec
ExceptionObject caught !

itk::ExceptionObject (0x563a086a0fa0)
Location: "unknown" 
File: /home/gdevenyi/projects/ITK/Modules/IO/NIFTI/src/itkNiftiImageIO.cxx
Line: 1988
Description: ITK ERROR: ITK only supports orthonormal direction cosines.  No orthonormal definition found!



    Start 1158: itkNiftiSmallVoxelsAffinePrecisionTest_fromitkNiftiImageIOTest2
2/2 Test #1158: itkNiftiSmallVoxelsAffinePrecisionTest_fromitkNiftiImageIOTest2 ...   Passed    0.03 sec

50% tests passed, 1 tests failed out of 2

Label Time Summary:
ITKIONIFTI    =   0.07 sec*proc (2 tests)

Total Test time (real) =   0.22 sec

The following tests FAILED:
        1157 - itkNiftiSmallVoxelsAffinePrecisionTest (Failed)
Errors while running CTest

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So itkNiftiImageIOTest2 just cannot be used for SmallVoxels_AffinePrecision.nii.gz, for some unknown reason, right?

Copy link
Contributor Author

@gdevenyi gdevenyi May 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So itkNiftiImageIOTest2 just cannot be used for SmallVoxels_AffinePrecision.nii.gz, for some unknown reason, right?

Correct.

The difference between the two sets of code appears to be

    itk::NiftiImageIO::Pointer io = itk::NiftiImageIO::New();
    auto                       imageReader = ImageReaderType::New();
    imageReader->SetImageIO(io);

Where the NiftiimageIO ImageIO is explicitly set in itkNiftiImageIOTest2

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gdevenyi Indeed, itkNiftiImageIOTest2 does explicitly set ImageIO, but it does also do a "read" without explicitly setting ImageIO. By calling itk::IOTestHelper::ReadImage:

input = itk::IOTestHelper::ReadImage<ImageType>(std::string(arg2));

Anyway, it seems to me that the CMake code of itkNiftiIOShouldSucceed was already incorrect. It says:

 itk_add_test(NAME itkNiftiIOShouldSucceed 
       COMMAND ITKIONIFTITestDriver itkNiftiImageIOTest2 
               ${ITK_TEST_OUTPUT_DIR} itkNiftiIOShouldSucceed true DATA{${ITK_DATA_ROOT}/Input/LittleEndian.hdr,LittleEndian.mhd,LittleEndian.img}) 

itk_add_test(NAME itkNiftiIOShouldSucceed
COMMAND ITKIONIFTITestDriver itkNiftiImageIOTest2
${ITK_TEST_OUTPUT_DIR} itkNiftiIOShouldSucceed true DATA{${ITK_DATA_ROOT}/Input/LittleEndian.hdr,LittleEndian.mhd,LittleEndian.img})

The arguments of itk_add_test don't seem to match correctly with arg1, arg2, and prefix at

if (argc != 4)
return EXIT_FAILURE;
char * arg1 = argv[1];
char * arg2 = argv[2];
char * prefix = argv[3];
int test_success = 0;
using ImageType = itk::Image<short, 3>;
using ImagePointer = ImageType::Pointer;
if ((strcmp(arg1, "true") == 0) && WriteNiftiTestFiles(prefix) == -1)

When I run it locally, arg1 == "itkNiftiIOShouldSucceed", arg2 == "true", and prefix == "C:/X/bin/I/ITK/ExternalData/Testing/Data/Input/LittleEndian.hdr". While arg1 should have been the boolean ("true" or "false"), and arg2 should have been the input file name. Right?

I think itkNiftiSmallVoxelsAffinePrecisionTest should be more like the following, if you would still like to use the existing itkNiftiImageIOTest2:

itk_add_test(NAME itkNiftiSmallVoxelsAffinePrecisionTest
      COMMAND ITKIONIFTITestDriver itkNiftiImageIOTest2
              ${ITK_TEST_OUTPUT_DIR} true DATA{${ITK_DATA_ROOT}/Input/SmallVoxels_AffinePrecision.nii.gz} SomeSpecificPrefix)

I don't know what exactly the prefix ("SomeSpecificPrefix") should be. But it should be the last argument to itk_add_test.

My two cents, Niels

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
5afad4b7b6fe872a896ed7eb555a572dc893e8074555973aca1bff3d2c12f6549a6d79b88d8c869c4e0b514de8b049a92b74fc09765fa82cbb21494f4e6608d6
52 changes: 52 additions & 0 deletions Modules/IO/NIFTI/test/itkNiftiImageIOTest13.cxx
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
/*=========================================================================
*
* Copyright NumFOCUS
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0.txt
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*=========================================================================*/

#include "itkNiftiImageIOTest.h"

// Test to read small voxel NIFTI file which was triggering numerical instability
// in IsAffine loading code
int
itkNiftiImageIOTest13(int argc, char * argv[])
{
if (argc != 2)
{
return EXIT_FAILURE;
}

constexpr unsigned int Dimension = 3;

using PixelType = float;
using ImageType = itk::Image<PixelType, Dimension>;

try
{
using ReaderType = itk::ImageFileReader<ImageType>;
ReaderType::Pointer reader = ReaderType::New();
reader->SetFileName(argv[1]);
reader->Update();
ImageType::Pointer image = reader->GetOutput();
Comment on lines +38 to +42
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider using itk::ReadImage instead. Just one line of code should so it 😃

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe some extra checks here to see that the read image is as expected...? Or at least, check that it is a valid image?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I do not have any C++ object-oriented black magic to anything beyond copying the ITK official example, https://examples.itk.org/src/io/imagebase/readanimage/documentation
  2. This test is to explicitly see if the NiftiIO isAffine check fails on the random image I had in my collection which was causing issues

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is to explicitly see if the NiftiIO isAffine check fails on the random image I had in my collection which was causing issues

I guess you could still add some checks that the read image is valid, for example:

if (image == nullptr)
{
  std::cerr << "Error: The read image is null!\n";
  return EXIT_FAILURE;
}

if (image->GetBufferPointer() == nullptr)
{
  std::cerr << "Error: the buffer of the read image is null!\n";
  return EXIT_FAILURE;
}

if (image->GetBufferedRegion().GetNumberOfPixels() == 0)
{
  std::cerr << "Error: The read image has no pixels!\n";
  return EXIT_FAILURE;
}

Right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gdevenyi Did you still consider adding the extra checks that I suggested here? In order to ensure that the reading does actually yield a valid image?

if (image == nullptr)
{
  std::cerr << "Error: The read image is null!\n";
  return EXIT_FAILURE;
}

if (image->GetBufferPointer() == nullptr)
{
  std::cerr << "Error: the buffer of the read image is null!\n";
  return EXIT_FAILURE;
}

if (image->GetBufferedRegion().GetNumberOfPixels() == 0)
{
  std::cerr << "Error: The read image has no pixels!\n";
  return EXIT_FAILURE;
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be condiserably abbreviated via itk::ReadImage and ITK_TRY_EXPECT_NO_EXCEPTION. We can do that in a follow-up PR. Let's merge this, as it has dragged on for quite a long time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both done in #3448.

}
catch (const itk::ExceptionObject & err)
{
std::cerr << "ExceptionObject caught !" << std::endl;
std::cerr << err << std::endl;
return EXIT_FAILURE;
}

return EXIT_SUCCESS;
}