RXCROPMATURITY_Lm61.f09_g17.IHistClm50BgcCrop.derecho_intel.clm-cropMonthOutput fails in ctsm5.1.dev175

[ekluzek]

Brief summary of bug

RXCROPMATURITY_Lm61.f09_g17.IHistClm50BgcCrop.derecho_intel.clm-cropMonthOutput

fails in ctsm5.1.dev175, possibly earlier

General bug information

CTSM version you are using: ctsm5.1.dev175

Does this bug cause significantly incorrect results in the model's science? No

Configurations affected: Just the system test

Details of bug

Looks like case.setup isn't done at the point it's needed which means the .env_mach_specific.sh in the $CASEDIR..gddgen doesn't exist, which is at least one of the problems.

Since the

. $CASEDIR.gddgen/.env_mach_specific.sh; CONDA_PREFIX=; conda activate npl && python3 $CTSMROOT/tools/modify_input_files/fsurdat_modifier $CASEDIR.gddgen/modify_fsurdat_allcropseverywhere.cfg

command run is complex it would be good to have some checking proceed it to check for the existence of both the
.env_mach_specific.sh file and the modify_fsurdat config file. It looks like that would be the only things that could be done to break it up into more understandable pieces.

Important output or errors that show the problem

TestStatus.log

 ---------------------------------------------------
2024-04-05 12:32:49: Exception during run:
Command '. /glade/derecho/scratch/erik/ctsm5.1.dev175/cime/scripts/RXCROPMATURITY_Lm61.f09_g17.IHistClm50BgcCrop.derecho_intel.clm-cropMonthOutput.20240405_122456_wckjn4.gddgen/.env_mach_specific.sh; CONDA_PREFIX=;  conda activate npl &&  python3 /glade/derecho/scratch/erik/ctsm5.1.dev175/tools/modify_input_files/fsurdat_modifier /glade/derecho/scratch/erik/ctsm5.1.dev175/cime/scripts/RXCROPMATURITY_Lm61.f09_g17.IHistClm50BgcCrop.derecho_intel.clm-cropMonthOutput.20240405_122456_wckjn4.gddgen/modify_fsurdat_allcropseverywhere.cfg ' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/glade/derecho/scratch/erik/ctsm5.1.dev175/cime_config/SystemTests/systemtest_utils.py", line 52, in run_python_script
    subprocess.run(
  File "/glade/u/apps/opt/conda/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '. /glade/derecho/scratch/erik/ctsm5.1.dev175/cime/scripts/RXCROPMATURITY_Lm61.f09_g17.IHistClm50BgcCrop.derecho_intel.clm-cropMonthOutput.20240405_122456_wckjn4.gddgen/.env_mach_specific.sh; CONDA_PREFIX=;  conda run -n npl python3 /glade/derecho/scratch/erik/ctsm5.1.dev175/tools/modify_input_files/fsurdat_modifier /glade/derecho/scratch/erik/ctsm5.1.dev175/cime/scripts/RXCROPMATURITY_Lm61.f09_g17.IHistClm50BgcCrop.derecho_intel.clm-cropMonthOutput.20240405_122456_wckjn4.gddgen/modify_fsurdat_allcropseverywhere.cfg ' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/glade/derecho/scratch/erik/ctsm5.1.dev175/cime/CIME/SystemTests/system_tests_common.py", line 282, in run
    self.run_phase()
  File "/glade/derecho/scratch/erik/ctsm5.1.dev175/cime_config/SystemTests/rxcropmaturity.py", line 138, in run_phase
    self._run_fsurdat_modifier()
  File "/glade/derecho/scratch/erik/ctsm5.1.dev175/cime_config/SystemTests/rxcropmaturity.py", line 281, in _run_fsurdat_modifier
    stu.run_python_script(
  File "/glade/derecho/scratch/erik/ctsm5.1.dev175/cime_config/SystemTests/systemtest_utils.py", line 64, in run_python_script
    subprocess.run(
  File "/glade/u/apps/opt/conda/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '. /glade/derecho/scratch/erik/ctsm5.1.dev175/cime/scripts/RXCROPMATURITY_Lm61.f09_g17.IHistClm50BgcCrop.derecho_intel.clm-cropMonthOutput.20240405_122456_wckjn4.gddgen/.env_mach_specific.sh; CONDA_PREFIX=;  conda activate npl &&  python3 /glade/derecho/scratch/erik/ctsm5.1.dev175/tools/modify_input_files/fsurdat_modifier /glade/derecho/scratch/erik/ctsm5.1.dev175/cime/scripts/RXCROPMATURITY_Lm61.f09_g17.IHistClm50BgcCrop.derecho_intel.clm-cropMonthOutput.20240405_122456_wckjn4.gddgen/modify_fsurdat_allcropseverywhere.cfg ' returned non-zero exit status 1.

[ekluzek]

@samsrabin is this something you could take on?

[samsrabin]

Sure, will do. It's not urgent, though.

[samsrabin]

As part of fixing this, will need to resolve #2522 (Remove hard-coded /glade/campaign reference in generate_gdds.py).

[slevis-lmwg]

closing manually as done