Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IODA Converter with In-Situ Observations Concatenation and ObsError Inflation #1472

Merged
merged 15 commits into from
Feb 13, 2025

Conversation

apchoiCMD
Copy link
Collaborator

@apchoiCMD apchoiCMD commented Jan 29, 2025

Description

This PR shows that an IODA Converter with Marine In-Situ Observations Concatenation and ObsError Inflation Capabilities from the table that shows various types of in-situ obs ("x" means that the variable is not provided)
Type Variables Argo Bathy Glider Tesac Tropical xbtctd drifter trkob
MetaData dateTime - - - - - - - -
- depth - - - - - - $\color{red}{\textsf{x}}$ $\color{red}{\textsf{x}}$
- latitude - - - - - - - -
- longitude - - - - - - - -
- oceanbasin - - - - - - - -
ObsError waterTemperature
salinity
-
-
-
$\color{red}{\textsf{x}}$
- - - - -
$\color{red}{\textsf{x}}$
-
ObsValue waterTemperature
salinity
-
-
-
$\color{red}{\textsf{x}}$
- - - - -
$\color{red}{\textsf{x}}$
-
PreQC waterTemperature
salinity
-
-
-
$\color{red}{\textsf{x}}$
- - - - -
$\color{red}{\textsf{x}}$
-
  • Double check on sail drone and axbt in-situ obs soon
  • Input yaml will be updated once test inputs are ready

Companion PRs

Nope

Issues

Resolves #1465

Automated CI tests to run in Global Workflow

  • atm_jjob
  • C96C48_ufs_hybatmDA
  • C96C48_hybatmaerosnowDA
  • C48mx500_3DVarAOWCDA
  • C48mx500_hybAOWCDA
  • C96C48_hybatmDA

@apchoiCMD
Copy link
Collaborator Author

Result is here by using ARGO @guillaumevernieres @AndrewEichmann-NOAA @JohnSteffen-NOAA

- gdas.t00z.insitu_profile_argo.2021032400.nc4
- gdas.t06z.insitu_profile_argo.2021032406.nc4
- gdas.t12z.insitu_profile_argo.2021032412.nc4
- gdas.t18z.insitu_profile_argo.2021032418.nc4
- gdas.t00z.insitu_profile_argo.2021032500.nc4
- gdas.t06z.insitu_profile_argo.2021032506.nc4
- gdas.t12z.insitu_profile_argo.2021032512.nc4
- gdas.t18z.insitu_profile_argo.2021032518.nc4
- gdas.t00z.insitu_profile_argo.2021032600.nc4

insitu_argo_redate_cycle

@apchoiCMD apchoiCMD marked this pull request as ready for review January 30, 2025 21:54
Copy link
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comments

utils/obsproc/InsituAll2ioda.h Show resolved Hide resolved
utils/obsproc/InsituAll2ioda.h Outdated Show resolved Hide resolved
utils/obsproc/InsituAll2ioda.h Outdated Show resolved Hide resolved
utils/obsproc/InsituAll2ioda.h Outdated Show resolved Hide resolved
@guillaumevernieres guillaumevernieres added the hercules-GW-RT Queue for automated testing with global-workflow on Hercules label Feb 11, 2025
@emcbot emcbot added hercules-GW-RT-Running Automated testing with global-workflow running on Hercules and removed hercules-GW-RT Queue for automated testing with global-workflow on Hercules labels Feb 11, 2025
@emcbot
Copy link

emcbot commented Feb 11, 2025

Automated GW-GDASApp Testing Results:
Machine: hercules

Start: Tue Feb 11 16:03:27 CST 2025 on hercules-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Tue Feb 11 16:42:19 CST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C48mx500_3DVarAOWCDA|C48mx500_hybAOWCDA|C96C48_hybatmDA
Tests:                                  *Failed*
Tests: Failed at Tue Feb 11 16:50:42 CST 2025
Tests: 97% tests passed, 1 tests failed out of 33
Tests: see output at /work2/noaa/da/role-da/CI/hercules/GDASApp/workflow/PR/1472/global-workflow/sorc/gdas.cd/build/log.ctest

@emcbot emcbot added hercules-GW-RT-Failed Automated testing with global-workflow failed on Hera and removed hercules-GW-RT-Running Automated testing with global-workflow running on Hercules labels Feb 11, 2025
@RussTreadon-NOAA
Copy link
Contributor

Hercules CI failure

Check log.ctest. test_gdasapp_util_insituall2ioda failed. Manually rerun this test with verbose output. Failure due to missing input yaml.

    Start 1598: test_gdasapp_util_insituall2ioda

1598: Test command: /work2/noaa/da/role-da/CI/hercules/GDASApp/workflow/PR/1472/global-workflow/sorc/gdas.cd/build/bin/gdas_obsprovider2ioda.x "../testinput/gdas_insituall2ioda.yaml"
1598: Environment variables:
1598:  OMP_NUM_THREADS=1
1598: Test timeout computed to be: 1500
1598: Exception: Cannot open ../testinput/gdas_insituall2ioda.yaml  (No such file or directory)
1598: terminate called after throwing an instance of 'eckit::CantOpenFile'
1598:   what():  Cannot open ../testinput/gdas_insituall2ioda.yaml  (No such file or directory)
1/1 Test #1598: test_gdasapp_util_insituall2ioda ...Subprocess aborted***Exception:  12.96 sec

@guillaumevernieres
Copy link
Contributor

Thanks @RussTreadon-NOAA .
@apchoiCMD , you probably forgot to create the link to the test yaml in the cmakelists.txt?

@apchoiCMD
Copy link
Collaborator Author

@RussTreadon-NOAA @guillaumevernieres Thanks for heads up and I will look into it as soon as possible.

@apchoiCMD apchoiCMD added the hercules-GW-RT Queue for automated testing with global-workflow on Hercules label Feb 12, 2025
@emcbot emcbot added hercules-GW-RT-Running Automated testing with global-workflow running on Hercules and removed hercules-GW-RT Queue for automated testing with global-workflow on Hercules labels Feb 12, 2025
@emcbot
Copy link

emcbot commented Feb 12, 2025

Automated GW-GDASApp Testing Results:
Machine: hercules

Start: Wed Feb 12 09:20:13 CST 2025 on hercules-login-1.hpc.msstate.edu
---------------------------------------------------
Build:                                 *SUCCESS*
Build: Completed at Wed Feb 12 09:57:04 CST 2025
---------------------------------------------------
Tests: ctest -j12 -R gdasapp -E atm_jjob|C96C48_ufs_hybatmDA|C96C48_hybatmaerosnowDA|C48mx500_3DVarAOWCDA|C48mx500_hybAOWCDA|C96C48_hybatmDA
Tests:                                 *SUCCESS*
Tests: Completed at Wed Feb 12 10:02:08 CST 2025
Tests: 100% tests passed, 0 tests failed out of 33

@emcbot emcbot added hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules and removed hercules-GW-RT-Running Automated testing with global-workflow running on Hercules labels Feb 12, 2025
Copy link
Contributor

@guillaumevernieres guillaumevernieres left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@guillaumevernieres guillaumevernieres removed the hercules-GW-RT-Failed Automated testing with global-workflow failed on Hera label Feb 13, 2025
@guillaumevernieres guillaumevernieres merged commit d5083d5 into develop Feb 13, 2025
11 checks passed
@guillaumevernieres guillaumevernieres deleted the feature/redate-insitu branch February 13, 2025 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hercules-GW-RT-Passed Automated testing with global-workflow successful on Hercules
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Redating and Error amplification for nc4 In-Situ obs
6 participants