No handlers could be found for logger "CIME.utils" on Titan

[mrnorman]

ACME master hash a4bc319

./create_newcase -compset FC5AV1C-04P2 -res ne16_ne16 -case test -mach titan -project stf006
cd test
./case.setup

[imn@titan-ext7:~/ACME/cime/scripts/get_data_downloaded] 8-) ./case.setup
Machine/Decomp/Pes configuration has already been done ...skipping
If an old case build already exists, might want to run 'case.build --clean' before building
Generating component namelists as part of setup
Creating component namelists
Running cam buildnml
CAM writing dry deposition namelist to drv_flds_in
Writing ocean component namelist to ./docn_in
CAM writing namelist to atm_in
Running clm buildnml
ERROR: Command: '/autofs/nccs-svm1_home2/imn/ACME/components/clm/cime_config/buildnml /autofs/nccs-svm1_home2/imn/ACME/cime/scripts/get_data_downloaded' failed with error 'No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
No handlers could be found for logger "CIME.utils"
** source directory does not exist: /autofs/nccs-svm1_home2/imn/ACME/components/clm/src/external_models/fates/main
ERROR clm.buildnml: /autofs/nccs-svm1_home2/imn/ACME/components/clm/bld/configure -comp_intf MCT -phys clm4_5 -phys clm4_5 -cppdefs -DMODAL_AER -usr_src /autofs/nccs-svm1_home2/imn/ACME/cime/scripts/get_data_downloaded/SourceMods/src.clm failed: 512'

[mrnorman]

This happens with FC5AV1C-L as well, and this is on Titan.

[mrnorman]

@rljacob , Continuing the discussion here for the Titan issue. I did the following, and the error still happens.

[imn@titan-ext1:~/ACME] 8-) git branch
  halldm2000/homme/sw_reorg
  jgfouca/cime/fixes_for_titan
* master
  mrnorman/cam5-physics/gpu-porting
  mrnorman/homme-tracers/preqx-acc-mpi-overlap
  mrnorman/homme-tracers/preqx-acc-mpi-overlap2
  mrnorman/homme/dynamics-merge
  mrnorman/machine-files/add-summitdev
  mrnorman/titan-machine/fix-mpich-bug
  mrnorman/titan-scripts/different-default-maxwalltime
  next
  nonhydro/homme/thetah+pese
[imn@titan-ext1:~/ACME] 8-) git fetch origin
X11 forwarding request failed on channel 0
remote: Counting objects: 27, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 27 (delta 22), reused 27 (delta 22), pack-reused 0
Unpacking objects: 100% (27/27), done.
From github.com:ACME-Climate/ACME
 * [new branch]      jayeshkrishna/mv_lite_man_pages -> origin/jayeshkrishna/mv_lite_man_pages
   048c0c5..8119bfd  jinyuntang/lnd/betr2 -> origin/jinyuntang/lnd/betr2
 * [new branch]      jqyin/lnd/fix-queryDefaultNamelist -> origin/jqyin/lnd/fix-queryDefaultNamelist
[imn@titan-ext1:~/ACME] 8-) git reset --hard origin/master
HEAD is now at a4bc319 Merge branch 'worleyph/machines/disable_perf_archiving_on_dev_systems' (PR #1568)
[imn@titan-ext1:~/ACME] 8-) git submodule update
[imn@titan-ext1:~/ACME] 8-) 

[minxu74]

@mrnorman I just checked out the latest master version (ea21bbf). I can create and set up your case without any problem. One of problem I noticed in your error message is that the FATE codes were not fetched into the acme external directory correctly. You may need to use "git submodule update --init". "--init" is required as the FATE was just added as an external module recently.

[jgfouca]

@mrnorman can this be closed? Looks identical to your other ticket.

[cameronsmith1]

I am getting a similar error on Cori. I am using git submodule update --init
The error I get (repeatedly) is:

No handlers could be found for logger "CIME.utils"  

From the log, it looks like it might is associated with the buildnml commands.

[PeterCaldwell]

I've been seeing tons of these issues as well. My understanding is that they aren't actually error messages and can be safely ignored. I think this info originated from @rljacob .

[AaronDonahue]

Yes, from a Slack conversation in the "General" channel on July 18th. I also get quite a few of these errors.

[jgfouca]

Yes, please ignore them. The latest CIME update will get rid of them entirely.

[rljacob]

Yes they are from buildnml scripts. They are calling CIME commands without POSIX-compliant arguments (using one dash when they should use 2). Ultimately that should be fixed but the error message will go away when PR #1643 is merged.

[rljacob]

Reopening because can not find the issue this was supposedly a duplicate of and it will be fixed in #1643