Skip to content

Commit

Permalink
Add utility to compare Fortran namelists (#1234)
Browse files Browse the repository at this point in the history
Often times it is necessary to compare Fortran namelists between a UFS-weather-model regression test and a global-workflow experiment, or in other example applications.

This PR adds a simple utility that loads two namelists and spits out the differences between them.  The differences are calculated as a departure from the first namelist.
This utility leverages `f90nml` (approved for use on WCOSS2)

The comparison is done as follows:
- Both namelists are loaded
- We loop over the keys of namelist 1.  We look for the same key in namelist 2.  If the key is found, the values are compared.  If the key is not found, a note is made that the key is undefined in namelist 2.
- Differences in the values are printed to screen.

```
❯❯❯ python3 compare_f90nml.py -h
usage: compare_f90nml.py [-h] -n namelist1 namelist2 [-r]

Compare two Fortran namelists and display differences

options:
  -h, --help            show this help message and exit
  -n namelist1 namelist2, --namelists namelist1 namelist2
                        name of two namelists to compare (namelist1 - namelist2) (default: None)
  -r, --reverse         reverse diff (namelist2 - namelist1) (default: False)
```

The `-r | --reverse` reverses the `namelists`.  This allows the user to use `namelist 2` as the reference.

If differences are found, they are shown as follows (examples of `input.nml` from the `control_p8` and `cpld_control_p8` regression tests of the ufs-weather-model)
```
❯❯❯ python3 compare_f90nml.py -n control_p8.nml cpld_control_p8.nml
comparing: control_p8.nml | cpld_control_p8.nml
-----------------------------------------------
atmos_model_nml:
  ccpp_suite : FV3_GFS_v17_p8 | FV3_GFS_v17_coupled_p8
fms_nml:
  domains_stack_size : 3000000 | 8000000
fv_core_nml:
  dnats : 0 | 2
gfs_physics_nml:
    min_seaice : 0.15 | 1e-06
  use_cice_alb : False | True
     nstf_name : [2, 1, 0, 0, 0] | [2, 0, 0, 0, 0]
        cplchm : False | True
        cplflx : False | True
        cplice : False | True
        cplwav : False | True
    cplwav2atm : False | True
```
  • Loading branch information
aerorahul authored Jan 11, 2023
1 parent 721e8ae commit 51ca1b3
Showing 1 changed file with 107 additions and 0 deletions.
107 changes: 107 additions & 0 deletions ush/compare_f90nml.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
#!/usr/bin/env python3

import json
import f90nml
from typing import Dict
from argparse import ArgumentParser, ArgumentDefaultsHelpFormatter


def get_dict_from_nml(filename: str) -> Dict:
"""
Read a F90 namelist and convert to a dictionary.
This method uses json to convert OrderedDictionary into regular dictionary
Parameters
----------
filename: str
Name of the F90 namelist
Returns
-------
dictionary: Dict
F90 namelist returned as a dictionary
"""
return json.loads(json.dumps(f90nml.read(filename).todict()))


def compare_dicts(dict1: Dict, dict2: Dict, path: str = "") -> None:
"""
Compare 2 dictionaries.
This is done by looping over keys in dictionary 1 and searching for them
in dictionary 2.
If a matching key is found, the values are compared.
If a matching key is not found, it is set to as UNDEFINED.
Note: A reverse match is not performed in this method. For reverse matching, use the -r option in the main driver.
Note: This is a recursive method to handle nested dictionaries.
Parameters
----------
dict1: Dict
First dictionary
dict2: Dict
Second dictionary
path: str (optional)
default: ""
key (if nested dictionary)
Returns
-------
None
"""

result = dict()
for kk in dict1.keys(): # Loop over all keys of first dictionary
if kk in dict2.keys(): # kk is present in dict2
if isinstance(dict1[kk], dict): # nested dictionary, go deeper
compare_dicts(dict1[kk], dict2[kk], path=kk)
else:
if dict1[kk] != dict2[kk]:
if path not in result:
result[path] = dict()
result[path][kk] = [dict1[kk], dict2[kk]]
else: # kk is *not* present in dict2
tt = path if path else kk
if tt not in result:
result[tt] = dict()
result[tt][kk] = [dict1[kk], 'UNDEFINED']

def _print_diffs(diff_dict: Dict) -> None:
"""
Print the differences between the two dictionaries to stdout
Parameters
----------
diff_dict: Dict
Dictionary containing differences
Returns
-------
None
"""
for path in diff_dict.keys():
print(f"{path}:")
max_len = len(max(diff_dict[path], key=len))
for kk in diff_dict[path].keys():
items = diff_dict[path][kk]
print(
f"{kk:>{max_len+2}} : {' | '.join(map(str, diff_dict[path][kk]))}")

_print_diffs(result)


if __name__ == "__main__":

parser = ArgumentParser(
description=("Compare two Fortran namelists and display differences (left_namelist - right_namelist)"),
formatter_class=ArgumentDefaultsHelpFormatter)
parser.add_argument('left_namelist', type=str, help="Left namelist to compare")
parser.add_argument('right_namelist', type=str, help="Right namelist to compare")
parser.add_argument('-r', '--reverse', help='reverse diff (right_namelist - left_namelist)',
action='store_true', required=False)
args = parser.parse_args()

nml1, nml2 = args.left_namelist, args.right_namelist
if args.reverse:
nml2, nml1 = nml1, nml2

dict1 = get_dict_from_nml(nml1)
dict2 = get_dict_from_nml(nml2)

msg = f"comparing: {nml1} | {nml2}"
print(msg)
print("-" * len(msg))
compare_dicts(dict1, dict2)

0 comments on commit 51ca1b3

Please sign in to comment.