AM Miscellaneous fixes to mimic Sundew behaviour

[andreleblanc11]

Place holder for all small fixes made to flowcb/gather/am.py or bulletin.py. These were mostly all picked up by my auditing script, comparing with the old NCP nodes.

Fixes made so far in PRs

Reject float types for Julian day value in getTime : #979
Add missing tokIsYear method from Sundew. Prevents some erronous bulletins not being filtered. : #973
Add fix to handle improperly parsed station names from getStation : #963
Properly handle trailing spaces at end of bulletin headers : #956
Move AM renamer to gather. Allows us to properly use sundew variable expansions. : #952
Lots of AM corrections (Renamer + AHL corrections in gather) : #946

[andreleblanc11]

With #997 's fix that got pushed last week, I'm hoping that those were the final changes to AM before NCP move.

There hasn't been any Sundew compatible bugs found for about two weeks now.

[andreleblanc11]

Found another bug. A BBB entry in the header was wrong and sr3 didn't filter it out like Sundew.

Sundew has a method that does this and more to verify the header. This should be likely added to sr3 -> https://github.com/MetPX/Sundew/blob/main/lib/bulletin.py#L601-L671

Not urgent. The sundew receiver downstream was able to filter the problem out so clients won't notice.

[petersilva]

BBB is OK, and should make it through... it depends on which header... I would only filter out invalid revision codes "AMENDMENT" is clearly wrong, but AMD, AAA, BBB, COR, are all OK.

UPDATE: nvmd... I understand you were referring to the header in general now, my bad.

[andreleblanc11]

Need a patch for MetPX/Sundew#20

[andreleblanc11]

Take note of MetPX/sr_insects#43 (comment)

• Some american bulletins *KWAL* and canadian SACN[34]* bulletins are failing

[andreleblanc11]

There hasn't been any bugs found in the last couple of months. A good sign.

[andreleblanc11]

2 months later, still no more bugs found

[andreleblanc11]

Some problems are now being found, now that I'm performing tests on our satnet senders.

• We're having problems parsing the station id of some american bulletins. From what I've found so far, this is because in sundew we had a list of WMO ID's that was being accumulated. https://github.com/MetPX/Sundew/blob/main/lib/wmoid.py

  • This was being called by the AM code and (I think) it looked for accumulated WMO ID's and would assign a station if it already exists https://github.com/MetPX/Sundew/blob/main/lib/bulletin.py#L373-L374

• I commented this out while writing the station fetcher as I didn't know what was its purpose..
  

  sarracenia/sarracenia/bulletin.py

  Lines 217 to 219 in 4a9f642

  	elif data[0][0:6] in ["SRCN40","SXCN40","SRMT60","SXAK50", "SRND20", "SRND30"]:
  	#elif data[0][0:6] in self.wmo_id:
  	station = premiereLignePleine.split()[0]

• Moving forward (some options)

  • We could implement a similar logic of sundew code to the sr3.
  • We could get the first element of the second line for all problematic bulletins coming in.
    • Would need to audit our data feeds and gather a list of all affected bulletins.

[petersilva]

I think this is about sending raw-inputs from GOES DCS we get from wallops to be picked up by CODECON. I think the CODECON stack has been replaced by DMS, and there is no need to forward this data on AM at all.