Expanded Test Suite Templated Input Models

[wbinventor]

This PR builds upon #248 with new InputSets for the test suite - in particular, an infinite homogeneous medium with 2-group data and 4x4 simple lattice problem (taken from our sample-inputs). This PR introduces 4 new tests:

• test_rings
• test_hom_inf_medium
• test_simple_lattice

In addition, the TestingHarness class' _get_results(...) method now makes it easy to add the number of FSRs, tracks, or segments to the database of results for regression tests. The "test_hom_inf_med," "test_pin_cell" and "test_simple_lattice" now all export FSR scalar fluxes to the regression results in addition to the eigenvalue.

[wbinventor]

I just pushed two new tests - "test_1d_gradient" and "test_2d_gradient" - which each utilize the homogeneous infinite medium template problem,

[wbinventor]

This PR now runs the test suite twice for 1 and 4 OpenMP threads. The number of threads used by the TrackGenerator is always only 1 in order to ensure reproducible FSRs. But the number of threads used by the Solver is varied. Also, this PR now includes a README for the test suite (adapted from that for OpenMC).

[wbinventor]

The latest commit adds "clang" as a secondary compiler to test along with "gcc." The Intel compilers are not yet supported on Travis, but it appears that may change based on the discussions here and here.

[wbinventor]

Alright, I added a bunch of new tests to this and think that I'm about ready to call it quits with this PR (save for any changes related to reviewer comments). Here is a summary of what I've added:

• Fixed source tests for the Solver::computeFlux(...) and Solver::computeSource(...) methods
• Tests for both FORWARD and for the ADJOINT calculation modes
• A templated PwrAssemblyInput based on the 17x17 assembly in "sample-input/benchmarks"
• A test for CMFD with the PwrAssemblyInput
• Updated "sample-input/ipython-notebook" to relect changes from recent PRs
• Added Geometry::setCmfd(...) to "thisown.i" so that Cmfd will not be garbage collected by Python
• Removed "clang" tests from Travis, but a user can easily toggle between compilers if they wish

[wbinventor]

After merging in #242 this PR failed for fixed source calculations. In order to fix this, I refactored the assignment of fixed sources by Cell and Material such that they are assigned at simulation runtime. In other words, a user may define a fixed source by Material or Cell with Solver::setFixedSourceByMaterial(...) or Solver::setFixedSourceByCell(...) at any point (even before tracks have been generated and FSRs assigned). Only when Solver::computeSource(...), Solver::computeFlux(...) or Solver::computeEigenvalue(...) is called are the sources by Material and/or Cell assigned to the appropriate FSRs in the array of fixed sources indexed by FSR and energy group.

[samuelshaner]

= 0 -> = False to be consistent throughout

[samuelshaner]

Looks like this still needs to be addressed.

[wbinventor]

Oops, it is covered now.

[samuelshaner]

any reason these lines are commented out?

[wbinventor]

Yes, at the moment it is not easy to install Intel compilers on Travis but this may change soon (see this discussion). The version of clang installed by apt-get on Ubuntu 12.04 is not new enough to support some of the C++11 features we need. However, if you want to run the test suite with icpc or clang on a personal machine or nsecluster, all you have to do is uncomment the appropriate lines here. So the reason for these lines is simply to show them as an example.

I will say that the test suite passes on my machine for clang, but fails for some tests with icpc on nsecluster. It appears that a few fluxes for some tests vary in the 6th decimal place in sequential mode, and many more vary when run in parallel, indicating that roundoff error is different with Intel than GNU.

[samuelshaner]

Could the flux results for tests/test_cmfd_pwr_assembly/results_true.dat just be hashed? There's quite a few values.

[samuelshaner]

ditto for tests/test_compute_flux/results_true.dat

[samuelshaner]

Any reasoning behind this number?

[wbinventor]

Nope - would you like a different value?

[samuelshaner]

I don't have a strong preference on any number so lets just keep it at 500.

[samuelshaner]

Is there any rule for when results should be hashed (or possibly abridged)?

[wbinventor]

@samuelshaner we should probably determine a standard for when and when not to hash the output. Right now the only results that are hashed are for "test_cmfd_pwr_assembly" (although you seemed to indicate you thought all of the values were included). At this early stage for the test suite I think we should shy away from hashing since it can be instructive to run the suite on different platforms (e.g., different compilers) and identify differences we never tracked before. This is more difficult to do when the results are hashed.

What do you think about hashing output strings that are > 1000 lines long? According to my estimates, such a long file would be about ~15 KB to store, but at least provide us an easy way to check results as we flesh out the suite.

[wbinventor]

This is now up-to-date.

[samuelshaner]

Oops, I meant to say tests/test_adjoint_simple_lattice/results_true.dat, which is 3,587 lines long. I think hashing or abridging results that are > 1000 lines is a good idea. What do you think about abridging instead of hashing, again because hashing doesn't give any info about how different the numbers are.

[wbinventor]

Ok I'm fine hashing the results for "test_adjoint_simple_lattice", as well as "test_forward_simple_lattice" which is the same length. I'm not sure about an option to abridge results, perhaps that can come with future PRs with bigger more complex test cases.

[wbinventor]

Now hashing the results for those two test cases.

[wbinventor]

Just a heads up that I accidentally added another commit to this PR to fix a few issues I identified today. I intended to submit these changes as a separate PR, but since I already committed them, perhaps they can just be reviewed here. These changes include the following:

• Removed Cell fills from the Geometry::toString() method to allow track files to be imported for geometries with different MGXS libraries / Materials
• Made neighbor cell optimizations for ray tracing optional. I've found the algorithm to build neighbor cells to be extremely slow for geometries processed with OpenCG's region differentiation algorithm and which may have O(10,000) Surfaces. Improving the performance for this will have to wait for another day.

[geogunow]

My opinion on hashing: we should only hash results we think should match exactly. For instance, if we are convinced that the eigenvalue should be an exact value then we should hash it. But if we think pin powers could be off by 0.001% and still be accurate then maybe we shouldn't hash the results. Another idea is to round the results to the degree to which we believe they should match and then hash that result. For instance, if we determine we want pin powers to match within 0.01% then we could round to the relevant decimal then hash the results.

[wbinventor]

@geogunow I think the debate for / against hashing also has to account for 1) whether the data to compare is larger than we'd like to track with Git and 2) whether having the raw data for comparison makes it more transparent and hence easier to debug tests (note that these two opposing forces are driving the debate over hashing tests results for OpenMC). At the moment, the TestingHarness truncates keff and the FSR fluxes to 6 decimal places. This is not to say that this is the best value, but provides us with a starting point for our testing infrastructure. The pin powers / FSR fission rates isn't yet even an option for the testing results but can be added as we build upon this.

[geogunow]

Is there a reason for the double *?

[wbinventor]

Nope, just a typo!

[wbinventor]

Is there anything that I've left undone in this PR? I'd like to merge this in before adding further to the test suite.

[samuelshaner]

I'm happy with this PR. The test suite is starting to look really nice!

[jeffhammond]

@wbinventor Since you cited my comments on using Intel compilers within Travis, which works via https://github.com/nemequ/icc-travis, you might also be interested in the newly released https://github.com/nemequ/pgi-travis .