Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update develop-ref after #1741 #1743

Merged
merged 92 commits into from
Apr 2, 2021
Merged
Changes from 1 commit
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
ebddb56
Start on write netcdf pickle alternative.
Jan 26, 2021
0fdbfdd
Write dataplane array.
Jan 26, 2021
6d46603
Start on read of netcdf as pickle alternative.
Jan 26, 2021
6fe4245
Create attribute variables.
Feb 2, 2021
644db21
Use global attributes for met_info attrs.
Feb 3, 2021
6594062
Add grid structure.
Feb 4, 2021
c6667e3
Read metadata back into met_info.attrs.
Feb 4, 2021
1e6eb9e
Convert grid.nx and grid.ny to int.
Feb 4, 2021
e005585
Rename _name key to name.
Feb 4, 2021
ab986ca
Removed pickle write.
Feb 4, 2021
760b690
Fixed write_pickle_dataplane to work for both numpy and xarray.
Feb 5, 2021
791ebf0
Use items() to iterate of key, value attrs.
Feb 5, 2021
c5f17e8
Write temporary text file.
Feb 13, 2021
d6142e8
Renamed scripts.
Feb 17, 2021
b39ca28
Changed script names in Makefile.am.
Feb 17, 2021
7cc2d77
Replaced pickle with tmp_nc.
Feb 17, 2021
df0db18
Fixed wrapper script names.
Feb 17, 2021
044c704
Test for attrs in met_in.met_data.
Feb 17, 2021
d798e9d
Initial version of read_tmp_point module.
Feb 18, 2021
8116e75
Added read_tmp_point.py to install list.
Feb 18, 2021
7b57715
Start on Python3_Script::read_tmp_point.
Feb 18, 2021
5502da9
Write MPR tmp ascii file.
Feb 18, 2021
961b4fc
Renamed to read_tmp_ascii to use for point point and MPR.
Feb 18, 2021
4c0963d
Renamed to read_tmp_ascii to use for point point and MPR.
Feb 18, 2021
91122be
Define Python3_Script::import_read_tmp_ascii_py.
Feb 19, 2021
fef8484
Call Python3_Script::import_read_tmp_ascii_py.
Feb 19, 2021
93e9762
Append MET_BASE/wrappers to sys.path.
Feb 20, 2021
44d8328
Finished implementation of Python3_Script::import_read_tmp_ascii_py.
Feb 20, 2021
3953aba
Call Python3_Script::read_tmp_ascii in python_handler.
Feb 20, 2021
25961d6
Revised python3_script::read_tmp_ascii with call to run, PyRun_String.
Feb 22, 2021
794e8fb
Return PyObject* from Python3_Script::run.
Feb 22, 2021
d569cfb
Restored call to run_python_string for now.
Feb 22, 2021
ab0f2c6
Per #1429, enhance error message from DataLine::get_item(). (#1682)
JohnHalleyGotway Feb 26, 2021
b2754b4
Feature 1429 tc_log second try (#1686)
JohnHalleyGotway Feb 26, 2021
a1aead4
Feature 1588 ps_log (#1687)
JohnHalleyGotway Feb 26, 2021
2ba6cd9
#1454 Disabled plot_data_plane_CESM_SSMI_microwave and plot_data_plan…
Feb 26, 2021
fe5f318
#1454 Moved NC attribute name to nc_utils.h
Feb 26, 2021
c1463b2
#1454 Corrected sanity checking for lat/lon projection based on the p…
Feb 26, 2021
ac4d7f9
#1454 Corrected sanity checking for lat/lon projection based on the p…
Feb 26, 2021
371dd2c
#1454 Corrected data.delta_lon
Mar 1, 2021
24c2bd8
#1454 Change bact to use diff instead of absolute value of diff
Mar 1, 2021
e224ce1
454 Deleted instea dof commenting out
Mar 1, 2021
c8e9049
454 Deleted instea dof commenting out
Mar 1, 2021
0f5366c
Merge pull request #1688 from dtcenter/feature_1454_nccf_grid_definition
hsoh-u Mar 1, 2021
996197c
Feature 1684 bss and 1685 single reference model (#1689)
JohnHalleyGotway Mar 2, 2021
40b57af
Per #1691, add met-10.0.0-beta4 release notes. (#1692)
JohnHalleyGotway Mar 2, 2021
23dc482
Updated Python documentation
jprestop Mar 3, 2021
9c9c54c
Per #1694, add VarInfo::magic_str_attr() to construct a field summary…
JohnHalleyGotway Mar 4, 2021
a16bebc
Per #1694, fixing 2 issues here. There was a bug in the computation o…
JohnHalleyGotway Mar 4, 2021
21e3eb7
Per #1694, just switching to consistent variable name.
JohnHalleyGotway Mar 5, 2021
0f84a5a
Just consistent spacing.
JohnHalleyGotway Mar 5, 2021
5f551d2
Merge pull request #1696 from dtcenter/bugfix_1694_grid_diag
davidfillmore Mar 5, 2021
f21b2e6
Added python3_script::import_read_tmp_ascii.
Mar 5, 2021
ba85a46
Merge branch 'develop' into feature_1319_no_pickle
Mar 5, 2021
31ae2e4
Restored read_tmp_ascii call.
Mar 5, 2021
f8becb9
Added lookup into ascii module.
Mar 5, 2021
9224844
Adding files for ReadTheDocs
jprestop Mar 5, 2021
1b41a0a
Adding .yaml file for ReadTheDocs
jprestop Mar 5, 2021
8382b33
Updated path to requirements.txt file
jprestop Mar 5, 2021
0303f1f
Updated path to conf.py file
jprestop Mar 5, 2021
99a6363
Removing ReadTheDocs files and working in separate branch
jprestop Mar 5, 2021
b0c8813
Return PyObject* from read_tmp_ascii.
Mar 7, 2021
bd9ed77
Put point_data in global namespace.
Mar 7, 2021
b358bed
Remove temporary ascii file.
Mar 7, 2021
4506173
Added tmp_ascii_path.
Mar 7, 2021
d6ed4b9
Removed read_obs_from_pickle.
Mar 7, 2021
592c937
Trying different options for formats (#1702)
jprestop Mar 9, 2021
d80aafa
Per #1706, add bugfix to the develop branch. Also add a new job to un…
JohnHalleyGotway Mar 10, 2021
6ed8fc4
Feature 1471 python_grid (#1704)
JohnHalleyGotway Mar 10, 2021
0f08b74
Committing a fix for unit_python.xml directly to the develop branch. …
Mar 11, 2021
2b5e37f
Merge branch 'develop-ref' into develop
JohnHalleyGotway Mar 11, 2021
48bb906
Add *.dSYM to the .gitignore files in the src and internal_tests dire…
JohnHalleyGotway Mar 11, 2021
6568493
Replaced tmp netcdf _name attribute with name_str.
Mar 12, 2021
22f5e98
Append user script path to system path.
Mar 15, 2021
18fb169
Merge pull request #1699 from dtcenter/feature_1319_no_pickle
davidfillmore Mar 15, 2021
dee5d3b
Revert "Feature 1319 no pickle" (#1717)
JohnHalleyGotway Mar 15, 2021
92f0ff0
Fixed typos, added content, and modified release date format
jprestop Mar 17, 2021
40fce11
#1715 Initial release
Mar 18, 2021
1ac92d7
#1715 Do not combined if there are no overlapping beteewn TQZ and UV …
Mar 18, 2021
9a97135
#1715 Added pb2nc_compute_pbl_cape
Mar 18, 2021
edb124b
#1715 Added pb2nc_compute_pbl_cape
Mar 18, 2021
aefabdb
#1715 Reduced obs_bufr_var. Removed pb_report_type
Mar 18, 2021
b7fb7c1
#1715 Added a blank line for Error/Warning
Mar 18, 2021
7998d89
Merge pull request #1719 from dtcenter/bugfix_1715_pb2nc_seg_fault_wi…
hsoh-u Mar 18, 2021
8dbef78
Per #1725, return good status from TrackInfoArray::add() when using a…
JohnHalleyGotway Mar 19, 2021
5866b2a
Per #1705, update the threshold node heirarchy by adding a climo_prob…
JohnHalleyGotway Mar 19, 2021
1a9f73a
Bugfix 1716 develop perc_thresh (#1722)
JohnHalleyGotway Mar 19, 2021
8dfd7c0
Update pull_request_template.md
JohnHalleyGotway Mar 19, 2021
e2f77e4
Feature 1733 exc (#1734)
JohnHalleyGotway Mar 29, 2021
6055600
Bugfix 1737 develop little_r (#1739)
JohnHalleyGotway Mar 30, 2021
804b1ac
Feature GitHub actions (#1742)
jprestop Apr 1, 2021
853ad34
Feature 1575 large_diffs (#1741)
JohnHalleyGotway Apr 1, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Removed read_obs_from_pickle.
  • Loading branch information
David Fillmore committed Mar 7, 2021
commit d6ed4b9262e9bf925faa7b5071763dfc67c298c9
22 changes: 10 additions & 12 deletions met/src/tools/other/ascii2nc/python_handler.cc
Original file line number Diff line number Diff line change
@@ -56,7 +56,7 @@ PythonHandler::PythonHandler(const string &program_name) : FileHandler(program_n

{

use_pickle = false;
use_tmp_ascii = false;

}

@@ -81,13 +81,13 @@ for (j=1; j<(a.n()); ++j) { // j starts at one here, not zero

}

use_pickle = false;
use_tmp_ascii = false;

const char * c = getenv(user_python_path_env);

if ( c ) {

use_pickle = true;
use_tmp_ascii = true;

user_path_to_python = c;

@@ -230,7 +230,7 @@ bool PythonHandler::readAsciiFiles(const vector< ConcatString > &ascii_filename_

bool status = false;

if ( use_pickle ) status = do_pickle ();
if ( use_tmp_ascii ) status = do_tmp_ascii ();
else status = do_straight ();

return ( status );
@@ -319,18 +319,17 @@ return ( true );


//
// wrapper usage: /path/to/python wrapper.py pickle_output_filename user_script_name [ user_script args ... ]
// wrapper usage: /path/to/python wrapper.py tmp_output_filename user_script_name [ user_script args ... ]
//

bool PythonHandler::do_pickle()
bool PythonHandler::do_tmp_ascii()

{

int j;
const int N = user_script_args.n();
ConcatString command;
ConcatString path;
ConcatString pickle_path;
ConcatString tmp_ascii_path;
const char * tmp_dir = 0;
int status;
@@ -347,14 +346,13 @@ path << cs_erase
<< tmp_dir << '/'
<< tmp_base_name;

// pickle_path = make_temp_file_name(path.text(), 0);
tmp_ascii_path = make_temp_file_name(path.text(), 0);
tmp_ascii_path << ".txt";

command << cs_erase
<< user_path_to_python << ' ' // user's path to python
<< replace_path(write_tmp_ascii_wrapper) << ' ' // write_tmp_point.py
<< tmp_ascii_path << ' ' // pickle output filename
<< tmp_ascii_path << ' ' // temporary ascii output filename
<< user_script_filename; // user's script name

for (j=0; j<N; ++j) {
@@ -367,7 +365,7 @@ status = system(command.text());

if ( status ) {

mlog << Error << "\nPythonHandler::do_pickle() -> "
mlog << Error << "\nPythonHandler::do_tmp_ascii() -> "
<< "command \"" << command.text() << "\" failed ... status = "
<< status << "\n\n";

@@ -389,8 +387,8 @@ PyObject * obj = script.lookup_ascii(list_name);

if ( ! PyList_Check(obj) ) {

mlog << Error << "\nPythonHandler::do_pickle() -> "
<< "pickle object is not a list!\n\n";
mlog << Error << "\nPythonHandler::do_tmp_ascii() -> "
<< "tmp ascii object is not a list!\n\n";

exit ( 1 );

10 changes: 4 additions & 6 deletions met/src/tools/other/ascii2nc/python_handler.h
Original file line number Diff line number Diff line change
@@ -50,9 +50,9 @@ class PythonHandler : public FileHandler
static string getFormatString() { return "python"; }


bool use_pickle;
bool use_tmp_ascii;

ConcatString user_path_to_python; // if we're using pickle
ConcatString user_path_to_python; // if we're using temporary ascii

ConcatString user_script_filename;

@@ -68,15 +68,13 @@ class PythonHandler : public FileHandler

virtual bool readAsciiFiles(const vector< ConcatString > &ascii_filename_list);

bool do_pickle ();
bool do_straight (); // straight-up python, no pickle
bool do_tmp_ascii();
bool do_straight (); // straight-up python, no temporary ascii

void load_python_obs(PyObject *); // python object is list of lists


bool read_obs_from_script (const char * script_name, const char * variable_name);

bool read_obs_from_pickle (const char * pickle_name, const char * variable_name);

};