Merge pull request #1428 from billsacks/move_unit_tests

Move unit tests to scripts/fortran_unit_testing
Moves unit testing infrastructure from tools/unit_testing to
scripts/fortran_unit_testing.

Rationale: tools are supposed to be optional tools, whereas the Fortran
unit tests are a more required part of the testing workflow.

Also cleans up python in fortran_unit_testing directory:

Remove dead code (which has been replaced by use of the standard CIME
utilities)

Clean up pylint problems

Also updates some documentation

Test suite: scripts_regression_tests on yellowstone
Test baseline: n/a
Test namelist changes: n/a
Test status: bit for bit

Fixes #1423
Fixes #1424

User interface changes?: Moves run_tests.py to scripts/fortran_unit_testing

Code review: jedwards

tools/unit_testing/README → scripts/fortran_unit_testing/README

@@ -1,20 +1,10 @@
 -*- mode:org -*-
 #+startup: showall
 
-Quick guide to the CESM unit testing framework
+Quick guide to the CIME unit testing framework
 
 * Requirements
 
-  - Machines :: Basic compiler information is retrieved from
-                config_compilers.xml in Machines. Even if running on an
-                unsupported machine, it is necessary to have this file to
-                get some basic information about the compiler.
-
-                As of this writing, it is possible to run with Machines,
-                this directory, the CMake module directory, and the
-                directory that specifies tests. A full CESM checkout is
-                more convenient but not needed.
-
   - Python 2.6 or later :: Required for scripts. Python 3.2 is also known
        to work.
 
@@ -40,31 +30,19 @@ Quick guide to the CESM unit testing framework
 ** Quick start
 
   In the simplest case, you may want to run tests from a single directory.
-  In that case, there are just three pieces of information that you usually
+  In that case, there are just two pieces of information that you usually
   need to specify:
 
   1. The location of the directory where you want to build and run the
      tests ("BUILD_DIR").
 
   2. The location of the directory specifying the test via a
      CMakeLists.txt file ("TEST_SPEC_DIR").
-
-  3. The compiler vendor ("COMPILER").
+     (Optional: Defaults to the current directory)
 
   The following command can run the tests:
 
-    run_tests.py --build-dir=BUILD_DIR --test-spec-dir=TEST_SPEC_DIR \
-        --compiler=COMPILER
-
-  The current compiler options are:
-
-  - GNU
-  - IBM
-  - Intel
-  - NAG
-  - PGI
-
-  COMPILER defaults to GNU.
+    run_tests.py --build-dir=BUILD_DIR --test-spec-dir=TEST_SPEC_DIR
 
   If you change a source file, you can run the same command again, and
   run_tests.py will do the bare minimum of work necessary to re-run the
@@ -79,18 +57,9 @@ Quick guide to the CESM unit testing framework
 
    Some particularly useful options are as follows:
 
-   + --build-type :: Setting this option to "CESM" will build the tests
-                     with CESM compiler options (or the most similar
-                     pFUnit-compatible flags). Setting this option to
-                     "CESM_DEBUG" will build with CESM's DEBUG mode
-                     options.
-
    + --clean :: Cleans the directory and re-runs both CMake and make when
                   running the tests.
 
-   + -R :: This option allows you to specify a regular expression; only
-             tests with names matching that expression will run.
-
    + -v :: Verbose output, including the commands used to compile, any
            compiler warnings, and CTest output from tests that pass. To get
            CTest output from tests that fail, set the environment variable
@@ -103,34 +72,41 @@ Quick guide to the CESM unit testing framework
 
 * Defining new tests using CMake
 
-  This README focuses on integration between the python scripts here, the
-  CMake modules, and CESM Machines. Further information, such as detailed
-  APIs, can be found in the documentation for pFUnit and the CMake modules
-  themselves.
+  This README focuses on integration between the python scripts here,
+  the CMake modules, and machine information. Further information, such
+  as detailed APIs, can be found in the documentation for pFUnit and the
+  CMake modules themselves.
 
   If you are new to this system, it is *highly* recommended that you look
   through the examples first.
 
-** CIME_utils - run_tests.py interface
+** CIME_initial_setup and CIME_utils - run_tests.py interface
 
-   The following CMake snippet will include the CESM utilities module. The
+   The following CMake snippet will include the CIME utilities module. The
    variable ${CIME_CMAKE_MODULE_DIRECTORY} is defined by run_tests.py, or
    by hand if you choose not to use run_tests.py and instead invoke cmake
    directly.
 
      list(APPEND CMAKE_MODULE_PATH ${CIME_CMAKE_MODULE_DIRECTORY})
+     include(CIME_initial_setup)
+     project(cime_tests Fortran C)
      include(CIME_utils)
 
+   The project name does not need to be 'cime_tests'. The key point of
+   the above snippet is that you need to include CIME_initial_setup
+   before the project line, and CIME_utils after the project line.
+
    CIME_utils processes a few options set by run_tests.py (e.g.
    "USE_COLOR"), and includes all of the following modules as well.
    Projects that do development without run_tests.py may choose to include
    only the modules below that they need.
 
-** Compilers - CESM compiler options
+** Compilers - CIME compiler options
 
    This module is also part of the run_tests.py interface; the primary
-   purpose is to read in flags generated from CESM's config_compilers.xml
-   file. However, it's also a catch-all for compiler-specific information.
+   purpose is to read in flags generated from the config_compilers.xml
+   file. However, it's also a catch-all for compiler-specific
+   information.
 
    The details of this module shouldn't be important to most users, most of
    the time, but it does provide one utility function that you may want to

tools/unit_testing/python/test_xml_test_list.py → scripts/fortran_unit_testing/python/test_xml_test_list.py

@@ -58,7 +58,7 @@ class TestSuitesFromXML(unittest.TestCase):
     """Tests for the suites_from_xml function."""
 
     def check_spec_list(self, xml_str, names, directories,
-                        known_paths={}, labels=None):
+                        known_paths=None, labels=None):
         """Check that a spec list matches input names and directories.
 
         This is used by the following tests to do the dirty work of making
@@ -129,7 +129,6 @@ def test_multiple_suites(self):
 
     def test_path_relative_to_known(self):
         """suites_from_xml handles a relative_to directory attribute."""
-        from os.path import abspath
 
         xml_str = """
 <root>
@@ -145,7 +144,6 @@ def test_path_relative_to_known(self):
 
     def test_path_with_whitespace(self):
         """suites_from_xml handles a directory with whitespace added."""
-        from os.path import abspath
 
         xml_str = """
 <root>
@@ -160,7 +158,6 @@ def test_path_with_whitespace(self):
 
     def test_path_with_label(self):
         """suites_from_xml handles a directory with a label correctly."""
-        from os.path import abspath
 
         xml_str = """
 <root>

tools/unit_testing/python/xml_test_list.py → scripts/fortran_unit_testing/python/xml_test_list.py

@@ -58,7 +58,7 @@ def __iter__(self):
         """
         return ( (l, d) for l, d in zip(self.labels, self.directories) )
 
-def suites_from_xml(xml_tree, known_paths={}):
+def suites_from_xml(xml_tree, known_paths=None):
     """Generate test suite descriptions from XML.
 
     Returns a TestSuiteSpec for each suite description in the XML input.
@@ -77,6 +77,9 @@ def suites_from_xml(xml_tree, known_paths={}):
        the known_paths dict.
     """
 
+    if known_paths is None:
+        known_paths = {}
+
     elements = xml_tree.findall("suite")
 
     for elem in elements:

tools/unit_testing/run_tests.py → scripts/fortran_unit_testing/run_tests.py

@@ -4,7 +4,7 @@
 _CIMEROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "../..")
 sys.path.append(os.path.join(_CIMEROOT, "scripts", "Tools"))
 sys.path.append(os.path.join(_CIMEROOT, "scripts", "utils", "python"))
-sys.path.append(os.path.join(_CIMEROOT, "tools", "unit_testing", "python"))
+sys.path.append(os.path.join(_CIMEROOT, "scripts", "fortran_unit_testing", "python"))
 
 from standard_script_setup import *
 from CIME.BuildTools.configure import configure
@@ -20,7 +20,6 @@
 #=================================================
 from printer import Printer
 from shutil import rmtree
-from distutils.spawn import find_executable
 # This violates CIME policy - move to CIME/XML directory
 from xml.etree.ElementTree import ElementTree
 

scripts/lib/CIME/code_checker.py

@@ -26,7 +26,7 @@ def _run_pylint(on_file, interactive):
     cmd_options += " --init-hook='sys.path.extend((\"%s\",\"%s\",\"%s\"))'"%\
         (os.path.join(cimeroot,"scripts","lib"),
          os.path.join(cimeroot,"scripts","Tools"),
-         os.path.join(cimeroot,"tools","unit_testing","python"))
+         os.path.join(cimeroot,"scripts","fortran_unit_testing","python"))
 
     cmd = "%s %s %s" % (pylint, cmd_options, on_file)
     logger.debug("pylint command is %s"%cmd)

scripts/tests/scripts_regression_tests.py

@@ -196,7 +196,7 @@ def test_a_unit_test(self):
         test_dir = os.path.join(cls._testroot,"unit_tester_test")
         cls._testdirs.append(test_dir)
         os.makedirs(test_dir)
-        unit_test_tool = os.path.abspath(os.path.join(CIME.utils.get_cime_root(),"tools","unit_testing","run_tests.py"))
+        unit_test_tool = os.path.abspath(os.path.join(CIME.utils.get_cime_root(),"scripts","fortran_unit_testing","run_tests.py"))
         test_spec_dir = os.path.join(os.path.dirname(unit_test_tool),"Examples", "interpolate_1d", "tests")
         run_cmd_no_fail("%s --build-dir %s --test-spec-dir %s"\
                             %(unit_test_tool,test_dir,test_spec_dir))
@@ -214,7 +214,7 @@ def test_b_cime_f90_unit_tests(self):
         cls._testdirs.append(test_dir)
         os.makedirs(test_dir)
         test_spec_dir = CIME.utils.get_cime_root()
-        unit_test_tool = os.path.abspath(os.path.join(test_spec_dir,"tools","unit_testing","run_tests.py"))
+        unit_test_tool = os.path.abspath(os.path.join(test_spec_dir,"scripts","fortran_unit_testing","run_tests.py"))
 
         run_cmd_no_fail("%s --build-dir %s --test-spec-dir %s"\
                             %(unit_test_tool,test_dir,test_spec_dir))