Improve robustness of cam open/close file register code to fix issues with history output

[jimmielin]

Tag name (required for release branches):
Originator(s): @jimmielin

Description (include the issue title, and the keyword ['closes', 'fixes', 'resolves'] followed by the issue number):

• Fixes History crashes when there are more than 3 output files #332 (max_mdims used before defined)
• Fixes max_mdims in cam_hist_file: config_define_file is used before defined #331 (unassociated of%file_desc in cam_register_open_file leading to crash with >2 history files)

Describe any changes made to build system: N/A

Describe any changes made to the namelist: N/A

List any changes to the defaults for the input datasets (e.g. boundary datasets): N/A

List all files eliminated and why: N/A

List all files added and what they do: N/A

List all existing files that have been modified, and describe the changes:
(Helpful git command: git diff --name-status development...<your_branch_name>)

Fixes #331
M       src/history/cam_hist_file.F90

Fixes #332
M       src/utils/cam_abortutils.F90

If there are new failures (compared to the test/existing-test-failures.txt file),
have them OK'd by the gatekeeper, note them here, and add them to the file.
If there are baseline differences, include the test and the reason for the
diff. What is the nature of the change? Roundoff?

derecho/intel/aux_sima:

derecho/gnu/aux_sima:

If this changes climate describe any run(s) done to evaluate the new
climate in enough detail that it(they) could be reproduced:

CAM-SIMA date used for the baseline comparison tests if different than latest:

[jimmielin]

Notes on the fixes in cam_register_open_file

Updated pool reuse logic

Was

         allocate(open_files_pool%next)
         open_files_pool%next => open_files_pool

There appears to be a circular reference as a result. I reversed the direction to be

         open_files_pool => open_files_pool%next

so that the pool now points to the next (or null) pool entry

Updated insert to linked list logic

Was

      open_files_tail => of_new
      if (.not. associated(open_files_head)) then
         open_files_head => of_new
      end if

This could be a problem if the list has more than 2 entries and a new node C is inserted

before:
head -> A -> B
tail -> B

after:
head -> A -> B
tail -> C

The updated logic should be more robust (the tail assignment line is repeated intentionally for clarity):

      ! Add the registered file to the tail of the open files list
      if(associated(open_files_tail)) then
         open_files_tail%next => of_new
         open_files_tail      => of_new
      else
         open_files_head      => of_new
         open_files_tail      => of_new
      endif

[peverwhee]

thanks for fixing these, @jimmielin - it's clear that past me had a very tenuous grasp of those pointers!

[nusbaume]

Thanks for these bug fixes @jimmielin!

[jimmielin]

Regression tests passed subject to pre-existing failures, merging in

derecho intel
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_intel.cam-outfrq_held_suarez_derecho (Overall: PASS) details:
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_intel.cam-outfrq_kessler_derecho (Overall: PASS) details:
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_intel.cam-outfrq_tj2016_after_coupler_derecho (Overall: PASS) details:
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_intel.cam-outfrq_tj2016_before_coupler_derecho (Overall: PASS) details:
  SMS_Ln9.mpasa480_mpasa480.FKESSLER.derecho_intel.cam-outfrq_kessler_mpas_derecho (Overall: FAIL) details:
  SMS_Ln9.ne5pg3_ne5pg3_mg37.FCAM7.derecho_intel.cam-outfrq_se_cslam_analy_ic (Overall: PEND) details:
  SMS_Ln9.ne5pg3_ne5pg3_mg37.FHS94.derecho_intel.cam-outfrq_se_cslam (Overall: PASS) details:
  SMS_Ln9.ne5pg3_ne5pg3_mg37.FKESSLER.derecho_intel.cam-outfrq_se_cslam (Overall: PASS) details:
  SMS_Ln9.ne5pg3_ne5pg3_mg37.FTJ16.derecho_intel.cam-outfrq_se_cslam (Overall: PASS) details:

derecho gnu
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_held_suarez_derecho (Overall: PASS) details:
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_kessler_derecho (Overall: PASS) details:
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_tj2016_after_coupler_derecho (Overall: PASS) details:
  SMS_Ln2.ne3pg3_ne3pg3_mg37.FPHYStest.derecho_gnu.cam-outfrq_tj2016_before_coupler_derecho (Overall: PASS) details:
  SMS_Ln9.mpasa480_mpasa480.FKESSLER.derecho_gnu.cam-outfrq_kessler_mpas_derecho (Overall: FAIL) details:
  SMS_Ln9.ne5pg3_ne5pg3_mg37.FCAM7.derecho_gnu.cam-outfrq_se_cslam_analy_ic (Overall: PEND) details:
  SMS_Ln9.ne5pg3_ne5pg3_mg37.FHS94.derecho_gnu.cam-outfrq_se_cslam (Overall: PASS) details:
  SMS_Ln9.ne5pg3_ne5pg3_mg37.FKESSLER.derecho_gnu.cam-outfrq_se_cslam (Overall: PASS) details:
  SMS_Ln9.ne5pg3_ne5pg3_mg37.FTJ16.derecho_gnu.cam-outfrq_se_cslam (Overall: PASS) details: