Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature 1753 python_fill #1757

Merged
merged 3 commits into from
Apr 16, 2021
Merged

Feature 1753 python_fill #1757

merged 3 commits into from
Apr 16, 2021

Conversation

JohnHalleyGotway
Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway commented Apr 16, 2021

Pull Request Testing

  • Describe testing already performed for these changes:

    Tested the python script from @j-opatz on kiowa to confirm that it was broken but is now fixed. Here's log output from the tests which shows the correct max value now.
cd /d1/personal/johnhg/MET/MET_Help/opatz_data_20210415
./run_pdp.sh
DEBUG 1: Opening data file: PYTHON_NUMPY
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new Met2dDataFile object of type "FileType_Python_Numpy".
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object of type "FileType_Python_Numpy".
DEBUG 3: Calling /usr/local/python3/bin/python3 to run user's python script (./poe_fcst_embedded).
DEBUG 4: Writing temporary Python dataplane file:
DEBUG 4: 	/usr/local/python3/bin/python3 /d1/personal/johnhg/MET/MET_Help/opatz_data_20210415/MET-feature_1753_python_fill/met/share/met/wrappers/write_tmp_dataplane.py /tmp/tmp_met_nc_36314_0 ./poe_fcst_embedded /d1/projects/CPC_data/input/precip_poe_fcsts/2019/08/07/gefs-legacy-00z_rfcst-cal_precip_20190807_week2.nc:33
...
DEBUG 4: Reading temporary Python dataplane file: /tmp/tmp_met_nc_36314_0
...
DEBUG 4:       plane min: 0.364983
DEBUG 4:       plane max: 0.814893

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

    The code for this PR is compiled on kiowa in:
    /d1/projects/MET/MET_pull_requests/met-10.0.0_beta5/feature_1753
    Please review the code changes and do whatever additional testing you'd like. Please run at verbosity 3 and 4 and notice that at level 4, the C++ code prints the python system command being executed.

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]
    None needed for this change, although we should update the docs for the change pickle to NetCDF temp files.

  • Do these changes include sufficient testing updates? [Yes]
    No additional testing needed.

  • Will this PR result in changes to the test suite? [No]

    If yes, describe the new output and/or changes to the existing output:

Pull Request Checklist

See the METplus Workflow for details.

  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s), Project(s), and Milestone
  • After submitting the PR, select Linked Issues with the original issue number.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

John Halley Gotway added 3 commits April 16, 2021 15:05
…When creating the variable to write the temp NetCDF file, we just need to pass through the fill value for the data. Also, make the script less verbose.
…MET. Update all 3 to print consistent log message when writing/reading the temp file. In particular, print the system command that is being executed at Debug(4) to make it easier to replicate python embedding problems that may arise.
@JohnHalleyGotway JohnHalleyGotway added this to the MET 10.0.0 milestone Apr 16, 2021
@JohnHalleyGotway JohnHalleyGotway linked an issue Apr 16, 2021 that may be closed by this pull request
21 tasks
@JohnHalleyGotway JohnHalleyGotway merged commit 8ab5382 into develop Apr 16, 2021
Copy link
Contributor

@j-opatz j-opatz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The results from a use case run were successful with the new code updates, producing the expected output that was present before the pickle switch (i.e beta3 output).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix user python embedding to handle the fill value correctly
2 participants