Merge pull request NOAA-EMC#147 from AndrewEichmann-NOAA/EXP-efso_fv3…

…_PR_util_EFSOI_scripts

GitHub Issue NOAA-EMC#118.  EFSOI additions to util/EFSOI-Utilities/scripts directory.

util/EFSOI_Utilities/scripts/README_fv3gfs

@@ -0,0 +1,22 @@
+
+This file, README_fv3gfs, describes the files in GSI/util/EFSOI_Utilities/scripts
+under the branch EXP-efso_fv3.  These files are expected to be merged into master
+shortly (as of 2020-07-30) 
+
+
+getefsiofiles.sh: obtains files necessary for gdaseupd task in global workflow,
+the modified version of which is the first step in the EFSOI process
+
+copyefsoifiles.sh: copies files from experiment to EFSOI staging directory after 
+gdasepos has run 
+
+display_osense.py: reads and plots sensitivity data for single cycle after reading
+from osense file generated by EFSOI executable
+
+osense.py: contains routine to read osense file
+
+convdata_codes.csv: codes needed to organize convetional data sources, derived from
+webpage (with the help of tools in pandas): 
+https://www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/table_2.htm 
+May need to be periodically updated
+

util/EFSOI_Utilities/scripts/display_osense.py

@@ -0,0 +1,49 @@
+import osense
+import pandas as pd
+
+
+( convdata, satdata )= osense.read_osense('osense_2017091000.dat.out')
+
+# creates a DataFrame with the mean of each satellite instrument
+#meanbyobtype = satdata.groupby('obtype').mean()
+
+meanbyobtype=satdata[['obtype','osense_kin','osense_dry','osense_moist']].groupby('obtype').mean()
+#cmeanbyobtype=convdata[['stattype','osense_kin','osense_dry','osense_moist']].groupby('stattype').mean()
+
+convcodes = pd.read_csv('convdata_codes.csv')
+
+# associate each data point with its source, by code
+# it would be more efficient to take the mean of the observation sensitivities by
+# code/stattype, but this way the mean is by the message column in the codes
+# (ADPUPA, AIRCRAFT, etc) to consolidate for simpler graphing. Taking the mean 
+# by code/stattype would break it down by data source more
+convbycodes=pd.merge(convdata,convcodes,how='left',left_on='stattype', right_on='code')
+
+convmean=convbycodes[['message','osense_kin','osense_dry','osense_moist']].groupby('message').mean()
+alltheobtypes=pd.concat([meanbyobtype,convmean])
+
+figuresize = (10,6)
+
+o_dry=alltheobtypes['osense_dry'].sort_values(ascending=False)
+dry_plot=o_dry.plot.barh(title = '2017091000 mean obs sensitivity, dry', figsize=figuresize);
+fig=dry_plot.get_figure()
+fig.savefig('obsense_dry.png',bbox_inches='tight')
+
+fig.clear()
+
+o_moist=alltheobtypes['osense_moist'].sort_values(ascending=False)
+moist_plot=o_moist.plot.barh(title = '2017091000 mean obs sensitivity, moist', figsize=figuresize);
+fig=moist_plot.get_figure()
+fig.savefig('obsense_moist.png',bbox_inches='tight')
+
+fig.clear()
+
+o_kin=alltheobtypes['osense_kin'].sort_values(ascending=False)
+kin_plot=o_kin.plot.barh(title = '2017091000 mean obs sensitivity, kinetic', figsize=figuresize);
+fig=kin_plot.get_figure()
+fig.savefig('obsense_kin.png',bbox_inches='tight')
+
+fig.clear()
+
+
+