Reenable and patch up HIP tests

[sethrj]

These should have been active for a long time.

[amandalund]

Good catch!

[sethrj]

Well this sucks, the HIP and CUDA codes are giving us different results for what should be identical device answers in the geo heuristic:

/var/jenkins/workspace/Celeritas_PR-711/test/celeritas/geo/HeuristicGeoTestBase.cc:93: Failure
Values in: avg_path
 Expected: this->reference_avg_path()
7 of 7 elements differ
by 0.001 relative error or 1.0000000000000001e-05 absolute error
i            EXPECTED           avg_path         Difference
0               58.02     56.656110952718 -0.0235072224626329
1               408.3    397.566004787006 -0.0262894812956022
2               274.9    264.307336236228 -0.0385327892461687
3               540.9    524.555623669406 -0.0302170019053313
4               496.3    481.338119354422 -0.0301468479661046
5                1134    1115.68753036875 -0.0161485622850543
6                1694    1647.72532660886 -0.0273168083772974

They're all a bit low, which is odd...

[sethrj]

@amandalund Possibly related to #715: I cherry-picked the GeoParams HIP onto v0.1 and v0.2; it passes on the former but fails on the latter. Remember also that the ORANGE "unit vector" assertion failures happened fairly consistently on v0.2 but never on v0.1.

At the very least, we might be able to bisect to find out when the geometry answers changed...

[amandalund]

Ah, that's a good data point... I agree, it would be useful to know when the HIP results changed, especially if it might be related to that other bug.

[amandalund]

Not sure why yet, but it looks like with HIP OrangeTrackView::cross_boundary() sometimes fails to find the post-crossing volume, and then because it's a release build sets the volume to be outside... which explains why the average paths are always a little smaller.

[sethrj]

OK, I git bisected and the first bad commit is #530...

$ git bisect start fd287ec35 ccc088df --
$ git bisect run ../git-bisect-run.sh

with the shell script based on the git bisect examples:

#!/bin/sh -x

BUILD=build-rocm-ndebug

if git cherry-pick --no-commit 1362c427 && ninja -C $BUILD
then
  GTEST_COLOR=1 \ctest --test-dir $BUILD -R celeritas/geo/Geometry -V
  STATUS=$?
else
  # Failed to build: return 'untestable'
  STATUS=125
fi

git reset --hard
exit $STATUS

[sethrj]

OK, I'm digging into this now and will have an answer later this evening. Reverting the testem3 to the non-background version doesn't fix it, and removing the "background intersect" doesn't fix it...

[sethrj]

These should be effectively equivalent but aren't... I'm thinking compiler bug?
83465ec

[amandalund]

A well-placed printf fixes it too.. there's definitely some undefined behavior.