pyIEM 1.22.0 Release

API Changes

• Add pyiem.reference.StationAttributes class to hold database attribute
  keys used.
• Discontinue raw SPS product text within the database.
• Ensure raw text products into WMOProduct end with a line feed.
• Move VTEC SE phenomena out of the default WFO jabber channels.
• Replace requests usage with httpx.
• Stop postprocessing the MapPlot figure into 8bit colorspace.

New Features

• Add allowed_as_list option to iemapp() helper to stop lists.
• Add pyiem.grid.nav with IEM grid information in a fancy form.
• Add MapPlot.imshow with some optimized panel plotting.
• Add maximum risk threshold within SPC outlook message (#969).
• Add pyiem.era5land with IEM grid reference information.
• Add pyiem.stage4 with grid reference information.
• Add support for plotting by FEMA Regions.
• Assign base WFO jabber channel to Tsunami Warnings (#978).
• Improvements for IEM netcdf grid navigation and handling.
• Include simple table of un-plotted states for MapPlot(sector="nws") #967.
• Introduce radar_ptype color ramp and draw_radar_ptype_legend for
  generating plots of HRRR ptype.

Bug Fixes

• Accomodate ancient LSRs using TRACE as the magnitude field.
• Add nounce to LSR autoplot link to prevent some ugliness around a NWS issue.
• Correct MapPanel GIS worldfile logic as upper left is center of grid cell.
• Ensure geometries going into masking helper are CCW, to mask outside of.
• Improve dev experience using ugc.UGCProvider (#980).
• Fix grid and affine definitions for IEMRE and MRMS.
• Properly check USDM service response prior to parsing it.
• Refine which MWW products are not defaulted into the main WFO Channels,
  those being VTEC codes SC, MF, and GL.
• Require valid wind speed and direction values going into windrose_utils.
• Support parsing CLI products circa 2007 with a bad space.
• Use more exact grid naviation for pyiem.prism, fixes off-by-one.

pyIEM 1.21.0 Release

API Changes

• Discontinue persisting text product into {mcd,mpd} table storage.
• Discontinue storage of text product into sigmets, use product_id instead.
• Raise IncompleteWebRequest exception when autoplot datetime parsing fails.

New Features

• Add _check_dueling_tropics VTEC check looking for TR+HU overlap (#930).
• Add IEMRE DOMAINS to support upcoming expansion.
• Add pyiem.grid.util.grid_smear to shift masked data to fill in missing.
• Allow iemapp(memcachexpire) to be a callable, called back with environ.
• Cross check WCNs against watch database storage for PDS status (#925).
• For iemapp if the memcachekey= callable returns None, the check of
  memcache is short circuited.
• Store product_id for PIREPs and do some faked AFOS assignments to help.
• Support SPC afternoon Day 3 outlook (cycle assigned as 20) (#936).
• Support VTEC significance addition of Extreme Heat (XH) (#953).

Bug Fixes

• Fix and refactor SPC saw parsing of watch replacement number.
• Pop kwargs fig on MapPlot.
• Use less confusing landing page at WPC for ERO.
• Use round instead of int to compute zonal stats grid navigation.

pyIEM 1.20.0 release

API Changes

• Change storage logic of SBW to not include CAN polygons in the case of
  a CAN/CON combination update (#888).
• Depend on docutils for reStructedText to HTML conversion.
• Don't generate jabber messages for LSR products with more than 4 LSRs (#901).
• util.get_autoplot_context can now raise IncompleteWebRequest.
• The VTEC/SBW database storage of raw NWS product text was discontinued.
  Instead, raw product_id values are stored, which can be used against IEM
  web services to get the raw text. (#857)

New Features

• Account for common AM/PM typos and off-by-one year timestamps in NWS Prods.
• Add backgroundcolor option to MapPlot.plot_values.
• Add comprehensive_climate_index and temperature_humidity_index.
• Add imgsrc_from_row helper for SPC outlook link generation.
• Add parse_commas option to webutil.ensure_list, so to allow comma
  delimited CGI params by default.
• For pydantic schema based iemapp, keys like wfo[] go to wfo.
• Introduce some pydantic based validators for web requests, experimental...
• Store product_signature in sbw table (requires iem-database schema update).
• Support SPC corrections for MCDs so to prevent database dups.

Bug Fixes

• Allow for pydantic field validation to work when there's legacy year vs
  year1 present.
• Correct generated LSR summary link to have timestamps in UTC.
• Correct handling of is_emergency when done within VTEC correction (#899).
• Correct SKC parsing within TAFs, level is now None in this case (#453).
• Don't allow CF6 data from "today" or the "future" to be parsed, if missing.
• Fix numeric instability with pyiem.plot.centered_bins (#871).
• Forward propagate sbw database storage of VTEC issuace (#862).
• Handle edge case with NWS text product having a polygon at the end without
  a trailing newline (ancient text).
• Improve NWS Text Product signature logic so to more generally match what
  looks like a signature (#865).
• Improved NWS MND header logic for timestamp retrieval.
• Make better life choices attempting to glean a NWS Text Product signature.
• Prevent situation in sbw database having polygon_begin > polygon_end (#862).
• Refactor VTEC/SBW storage logic to use vtec_year column and operate on the
  parent table (#863).
• Update link to new NWPS website.

pyIEM 1.19.0 Release

API Changes

• Drop pyiem.nwnformat as it is no longer used.
• Introduce pyiem.wmo.WMOProduct as a lightweight class for products that
  need little default processing. pyiem.nws.product.TextProduct inherits.
• Remove import * usage from pyiem.plot. This was a bad daryl mistake.
• Require pygrib as we add some grib processing.
• Require pyogrio for faster geopandas shapefile reading.

New Features

• Add pyiem.iemre.grb2iemre helper to reproject a grib message onto IEMRE.
• Add pyiem.util.archive_fetch helper to get IEM archived resources.
• Add VTEC storage of product_ids (#857).
• Sync NWS VTEC colors per IEM database cross-check review.
• Update bundled NWS zones database to 5 March 2024 release (#821).

Bug Fixes

• Account for a common autoplot context timestamp typo.
• Add a hacky gc.collect() within MapPlot#close attempting to workaround
  some matplotlib memory leaking.
• Constrain a VTEC database search for expiration times not infinitely into
  the future.
• Fix iemre.reproject2iemre to return a masked_array and handle an input
  masked array.
• Fix invalid rtrim() usage with removesuffix().
• Fix off-by-one when grid.zs hits a mask right at the border.
• Fix placement of GU, AK, HI, PR inset axes for sector="nws" plot.
• Improve setuptools_scm plumbing to avoid runtime import.
• Remove hackish website telemetry writing from a thread.
• Support a variant Trace value specified in LSRs.

pyIEM 1.18.0 Release

API Changes

• Add dependencies on nh3 and paste.
• Raise NewDatabaseConnectionFailure from pyiem.database.get_dbconn for
  all failures and stop emitting UserWarning.
• Raise UnknownStationException from pyiem.util.get_autoplot_context when
  an unknown station is provided.
• Repurpose LSR type internal code q for SNOW SQUALL.

New Features

• Add #{state}wx hash tags to NWS Local Storm Report social media posts.
• Add cursor_name option to database.get_dbconnc.
• Add pyiem.iemre.reproject2iemre helper to bring arbitrary grids onto IEMRE.
• Add pyiem.util.{delete,set}_property helpers.
• Introduce pyiem.webutil.iemapp decorator to do fancy things for IEM
  mod_wsgi apps and remove downstream boilerplate.

Bug Fixes

• [SHEF] Lines startig with ":" are double checked for slashes, before use.
• [SHEF] Ensure E message elements have a physical_element set.
• [SHEF] Fix DD modifier when we are crossing a month.
• [SHEF] Fix significant bug with Paired Values (ref SHEF Manual 7.4.6) not
  properly computing magnitude for encoded float values.
• [SHEF] Forgive a random nan that keeps appearing in RRSRAH.
• [SHEF] Prevent extraneous "/" added to contination .A and .E messages.
• [SHEF] Support 0.001 encoding of Trace values.
• Allow for calm percentage plotting space in windrose with rmax set (#798).
• Fix compute_wfos method for SAW processor.
• Make presence of tabs in CLI products more forgiving.
• Refactor SHEF parsing such to better handle situations with time modifiers
  being missing and enforce SHEFElement pydantic validation.

pyIEM 1.17.0 Release

API Changes

• MAJOR Migration of database connections to psycopg 3, introduce
  get_dbconnc to return a connection and dict_row cursor. Currently,
  get_dbconn will continue to return a default cursor, this may change in
  the future (#754).
• Default to psycopg 3 for sqlalchemy connections.
• Drop twython and remove pyiem.util.get_twitter as both are defunct after
  Twitter v1.1 removal.
• Fully drop backports.zoneinfo usage as we now require Python 3.9+.

New Features

• Add pyiem.util.get_dbconnc helper to return a connection + cursor.
• Add some AFOS product definitions for various NHC recon products.
• htmlgen/station_select was updated to include offline stations by default.
• Updated bundled NWS GIS to 19 Sep 2023 release (#734).

Bug Fixes

• Fix Jabber message generation for SPC Day4-8 Convective Outlooks.
• Misc improvements to NWS CLI product parsing.
• Require IBW tag confirmation before declaring a TOR,FFW product as an
  emergency, for products 2023 and onward.
• [SHEF] Account for corrupted timestamp generating a traceback.
• Upper case LSR typetext in Jabber channel usage.

pyIEM 1.16.0 Release

API Changes

• Convert marine LSR types that are used on land to legacy codes (#729).
• Dropped python=3.8 support/testing as xarray has moved on.
• Removed pyiem.cscap_utils as it was not maintained.
• Removed pyiem.twistedpg as it was a glorious hack and no longer needed.
• SPC/WPC outlook geometry is now the actual threshold specific geometry. A
  corresponding geometry_layers multipolygon now exists to hold the non cookie
  cutted polygon (#738).

New Features

• Add bling to jabber messages generated from PNS Damage Surveys.
• Add Impact Based Warning tags to jabber channels in the form of
  <phenomena>.<significance>.<tag> (#11).
• Add parser for Wind/Temps Aloft Forecast product (AWIPSID: FD) (#713).
• Add twitter_media for LSR jabber messages.
• Improve METAR parsing to better preserve integer `degF`` temperatures (#740).
• Persist Summary LSR product identifier to database (akrherz/pyWWA#150).
• Support type="state" for autoplot context.
• Support Guam for NEXRAD overlays.
• Updated bundled NWS counties/zones to something circa March 2023.
• Validate type=cmap for util.get_autoplot_context (#709).

Bug Fixes

• Correct VTEC is_emergency false positive spotted by Kyle Noël.
• Improve forgiveness of CLI parser some more. Never ends.
• Improve robustness of FLS impacts bullet search.
• Prevent CF6 data from the future.
• [SHEF] Workaround DV months offset, which is ill-defined anyway.
• Support LSRs with mile units.
• Fix LSR typetext comparisons and ensure database uppercase entries.

pyIEM 1.15.0 Release

API Changes

• Depend on package defusedxml for XML parsing.
• Depend on package twython.
• Depend on package pymemcache to support my hackish use of memcached.
• Quasi internal MetarCollective.wind_message was modified to also return the
  wind speed in knots.
• Refactor internal testing util.get_file_name helper.
• Refactor pyiem.util database methods to pyiem.database.

New Features

• Add flag (plot_convention) for windrose plots to change orientation of the
  bars. Engineers generally want bars oriented toward the direction the wind
  is blowing toward, not from (#680).
• Add isolated parameter to geoplot.plot_values to allow label collision
  to only be done against the plotted data in that iteration and not all data.
• Add fill_{rfc,cwsu} as available MapPlot methods.
• Add WPC XTEUS (national max/min temp) parser pyiem.nws.products.xteus.
• Fill out pyiem.data.reference.prodDefinitions based on what the IEM AFOS
  database has.
• Fix state border zorder and allow stateborderwidth kwarg.
• Jabber channels for METAR wind gust alerts were enhanced (#683).
• Generate a TextProduct.warning message for a VTEC product that should contain
  a polygon, but does not (#660).
• Introduce a natural earth background option for MapPlot (#304).
• Introduce hacky sector="spherical_mercator" that brings in ESRI basemaps
  for the background. My implementation stinks and will change (#304).
• Support cartopy_offlinedata version 0.20+.
• Support new CLI format diction from NWS Sacramento.
• Workaround autoplot context fun with mixed 3-4 character WFOs.

Bug Fixes

• Account for MapPlot custom domain that crosses anti-meridian (#655).
• Add GU "Guam" to pyiem.reference.state_names.
• Allow non-conforming DHMSG within SHEF.
• Cleanup and improve windrose title / time filtering logic (#663).
• Correct VTEC database accounting issue for emergencies (#676).
• Correct VTEC database partitioning for difficult event spanning years.
• Draw mask on all known sectored plots.
• Increased default pyiem.util.get_dbconn connect timeout to 30 seconds.
• Polish SHEF parsing some with better error message and account for ...
  headlines.
• Reduce needless lat/lon precision with Jabber messages (#656).
• Remove hard coded nobody database user for some internal API calls.
• Remove matplotlib colormap shim and require matplotlib>=3.5.
• [SHEF] Make station ids longer than 8 chars non-fatal.
• Support geos 3.11 (#633).
• Support increased range and emit ValueError for too large range for
  pyiem.plot.pretty_bins (#665).
• Update UP VTEC phenomena label to "Freezing Spray", remove ZY.

pyIEM 1.14.0

API Changes

• [SHEF] Permit one character SHEF physical codes (daryl gave up).
• [DS3505] Removed hacky metar/sql round trip code in NCEI ISH processor.
• [metarcollect] Break the internal API and storage for iemid and how
  iemaccess gets updated to allow for downstream changes.
• [NetworkTable] Change internal data structure from dict to list for station
  threading information (#645).
• [observation] Adjusted the constructor to allow some things to be optional
  and allow manual provision of iemid and tzname.

New Features

• Add support for upcoming Snow Squall Warning impact tags (#493).
• Attempt to resolve a NWS UGC code when storing a LSR to the database (#637).
• Enhance G-AIRMET processing to better define icing and not create airmet
  entries for multiple-level freezing airmets (#628).
• Generalize autoplot context parsing for params starting with _.
• Support Center Weather Advisory (CWA) that uses lat/lon points.
• Update bundled NWS Counties/Zones to circa 13 Sep 2022 (#648).

Bug Fixes

• [windrose] Cleanup the title and diagnostic for windrose plots.
• Allow trailing space in UGC encoding line (#652).
• Correct decoding of 12 UTC timestamp in MND header.
• Ensure CF6 weather codes go to the database verbatim without float conv.
• Fix G-AIRMET decoding of multiple freezing levels airmet.
• Workaround a specified 12 AM UTC timestamp in NWS text products.

pyIEM 1.13.0 release

API Changes

• Added sqlalchemy as a hard package requirement.
• pyiem.nws.products.saw no longer writes un-used SAW text to database.
• Refactor internal SHEF decoding such that TextProduct instance has a seat
  at the table when some acceptable parsing failures can happen. The functional
  form of process_messages_{a,b,e} is now TextProduct,str.
• Refactor internal VTEC API for creating jabber messages.
• Refactor windrose_utils and remove the positional argument cruft.
• The XMPP channels assigned to VTEC products that are CONtinues, EXPires, or
  CANcels was modified to append a -ACTion to the channel in the case of the
  channels prefixed by the phenomena.significance. This will cut down on
  the social media posts for products that are more mudane (#604).

New Features

• Add alpha control to draw_usdm.
• Add database storage of purge_time for VTEC products (#616).
• Add parsing support for CWSU Center Weather Advisory (#573).
• Add option to mcalc_feelslike to support mask_undefined.
• Add twitter_media link for generic text products that have a polygon (#586).
• Add limit_by_doy option to windrose_utils to allow a day of year limit.
• Add parser for SPC Watch Probabilities (WWP) product (#595).
• Add basic parser for SPC SEL Product.
• Allow pyiem.nws.nwsli instance to be subscriptable for iterop.
• Support passing linewidths to MapPlot.contourf.

Bug Fixes

• [SHEF] Catch invalid nan values.
• [windrose] Fixed logic bug when limit_by_doy was set and dates crossed
  1 January.
• Added some jitter to plotting labels for side-by-side WFOs.
• Fixed f-string formatting issue in SAW jabber message generator.
• Fixed missing ENH and MDT from spcpts.THRESHOLD_ORDER.
• Fixed jabber/twitter message generation for a VTEC product with multiple
  segments with the same vtec action (read tropical products).
• Heat index is now computed without the presence of wind info (#623).
• Improved logic behind pyiem.plot.util.pretty_bins, it no longer exactly
  returns the specified number of bins, but tries to do the right thing!
• Increase remark trimming for LSR tweets for more length safety.
• Stopped back-computing affected WFOs based on the UGCs found in a VTEC
  product. This was causing more confusion than good (#615).
• Support CHUT, PWT timezones in NWS products.