Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More path length issues with new system_tests_compare_two #1914

Closed
billsacks opened this issue Sep 21, 2017 · 5 comments
Closed

More path length issues with new system_tests_compare_two #1914

billsacks opened this issue Sep 21, 2017 · 5 comments

Comments

@billsacks
Copy link
Member

We have been running into a number of path length issues with the new system_tests_compare_two, where CIME_OUTPUT_ROOT for case2 looks like $CIME_OUTPUT_ROOT/TESTNAME/case2/TESTNAME, with the bld and run directories under there. For long testnames, this creates some very long source file names, which exceed various maximum limits for compilers. The latest issue I'm running into is that CMake tries to compile a test program in a subdirectory of the build directory, and ifort barfs when the abs path to this test source file is too long.

@billsacks
Copy link
Member Author

I propose fixing this by moving the location of the case2 bld directory.

So we'll still have the run directory here:

$CIME_OUTPUT_ROOT/TESTNAME/case2/TESTNAME/run

But we'll have the bld directory here:

$CIME_OUTPUT_ROOT/TESTNAME/case2/bld

I'll probably put in a symlink: $CIME_OUTPUT_ROOT/TESTNAME/case2/TESTNAME/bld -> $CIME_OUTPUT_ROOT/TESTNAME/case2/bld to make it easier for people to find the bld directory.

@billsacks
Copy link
Member Author

@fischer-ncar @ekluzek and @jedwards4b I think this will solve the cism intel build problems Chris ran into.

@jedwards4b
Copy link
Contributor

Maybe just change both EXEROOT and RUNDIR to take out the $CASE?
./xmlquery EXEROOT --no-resolve
EXEROOT: $CIME_OUTPUT_ROOT/$CASE/bld
yslogin6: ~/sandboxes/cesm2_0_alpha/cime/scripts/ERR.f19_g16_rx1.A.yellowstone_intel.GC.20170921_090015_k45txl
:) ./xmlquery RUNDIR --no-resolve
RUNDIR: $CIME_OUTPUT_ROOT/$CASE/run

I don't think that you need to add the link to the old path, no-one has been using this enough for that to be a difficulty.

@billsacks
Copy link
Member Author

@jedwards4b okay, I'm happy with that suggestion. So to be clear, we'd have:

Run directory:

$CIME_OUTPUT_ROOT/TESTNAME/case2/run

Bld directory:

$CIME_OUTPUT_ROOT/TESTNAME/case2/bld

Is that what you're thinking?

I'm thinking we'd still have case2's CIME_OUTPUT_ROOT be

$CIME_OUTPUT_ROOT/TESTNAME/case2, as it is now, but nothing should appear under $CIME_OUTPUT_ROOT/TESTNAME/case2/TESTNAME any more.

@jedwards4b
Copy link
Contributor

Yes - that's it. By default (if you don't using the --test-root option) the case directory will still appear in $CIME_OUTPUT_ROOT/TESTNAME/case2/TESTNAME

jedwards4b added a commit that referenced this issue Sep 25, 2017
Shorter paths for case2 using system_tests_compare_two
The recently-changed version of system_tests_compare_two nested case2's
bld and run directories under
$CIME_OUTPUT_ROOT/TESTNAME/case2/TESTNAME. Since the TESTNAME is often
long, this was leading to very long paths to some things in the bld
directory. This was causing some compilation errors.

With this PR, case2's bld and run directories appear in
$CIME_OUTPUT_ROOT/TESTNAME/case2/bld and
$CIME_OUTPUT_ROOT/TESTNAME/case2/run. This shortens the bld directory
paths enough to get some formerly-failing builds to pass. The change in
run directory is just for consistency.

I was hoping to do this just via setting EXEROOT and RUNDIR in
system_tests_compare_two, to avoid cluttering up create_clone with
arguments that are just needed for tests. But it turns out that we need
to do this in create_clone, because create_clone calls case_setup, which
creates the EXEROOT and RUNDIR directories.

Test suite: scripts_regression_tests on cheyenne
Also confirmed that the following tests build successfully:
ERP.f10_f10_musgs.I1850Clm45BgcCru.yellowstone_pgi
ERP_P60x2_Lm36.f10_f10_musgs.I2000Clm50BgcCrop.yellowstone_intel.clm-clm50cropIrrigMonth_interp
Test baseline: N/A
Test namelist changes: none
Test status: bit for bit

Fixes #1914

User interface changes?: none

Update gh-pages html (Y/N)?:

Code review: jedwards
jgfouca added a commit that referenced this issue Dec 8, 2017
…1914)

Add mkl_rt library needed for Skybridge

This links the Intel Math Kernel Library with vector functions needed
for the unit tests when compiling with the Intel compiler.

Test suite: scripts_regression_tests.py
Test status: all ok

Fixes #2080

Code review: @jgfouca

[BFB]

* origin/mfdeakin-sandia/pfunit/skybridge_test:
  Add mkl_rt library needed for Skybridge
jgfouca pushed a commit that referenced this issue Dec 8, 2017
Add configuration needed to run unit tests on skybridge. This uses the
pFUnit library built on skybridge and installed in the ccsm project
space on the SNL capacity clusters. Configuration also added to the
compiler sections for ghost and chama, but PR #1914 needs to be applied
to the configuration for these machines before unit tests will run on
ghost and chama.
jgfouca added a commit that referenced this issue Feb 23, 2018
…1914)

Add mkl_rt library needed for Skybridge

This links the Intel Math Kernel Library with vector functions needed
for the unit tests when compiling with the Intel compiler.

Test suite: scripts_regression_tests.py
Test status: all ok

Fixes #2080

Code review: @jgfouca

[BFB]

* origin/mfdeakin-sandia/pfunit/skybridge_test:
  Add mkl_rt library needed for Skybridge
jgfouca pushed a commit that referenced this issue Feb 23, 2018
Add configuration needed to run unit tests on skybridge. This uses the
pFUnit library built on skybridge and installed in the ccsm project
space on the SNL capacity clusters. Configuration also added to the
compiler sections for ghost and chama, but PR #1914 needs to be applied
to the configuration for these machines before unit tests will run on
ghost and chama.
jgfouca added a commit that referenced this issue Mar 13, 2018
…1914)

Add mkl_rt library needed for Skybridge

This links the Intel Math Kernel Library with vector functions needed
for the unit tests when compiling with the Intel compiler.

Test suite: scripts_regression_tests.py
Test status: all ok

Fixes #2080

Code review: @jgfouca

[BFB]

* origin/mfdeakin-sandia/pfunit/skybridge_test:
  Add mkl_rt library needed for Skybridge
jgfouca pushed a commit that referenced this issue Mar 13, 2018
Add configuration needed to run unit tests on skybridge. This uses the
pFUnit library built on skybridge and installed in the ccsm project
space on the SNL capacity clusters. Configuration also added to the
compiler sections for ghost and chama, but PR #1914 needs to be applied
to the configuration for these machines before unit tests will run on
ghost and chama.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants