multiple active webhooks from one repository generate many build completion messages

[mtmorgan]

e.g., two 'Received a valid push' messages received at Bioconductor/Contributions#968 (comment)

track_build_completion.log contains

ERROR: 01/10/2019 08:43:52 PM track_build_completion.py - failed to handle event: list.remove(x): x not in list.

which is from

spb_history/track_build_completion.py

Line 367 in b7ca04c

logging.error("failed to handle event: %s." % e)

Because the function returns immediately, the message is never marked as 'received' a few lines down. Presumably the message remains on the queue, and is processed again.

I am guessing that 'list.remove()' refers to

spb_history/track_build_completion.py

Line 57 in b7ca04c

build_counter.setdefault(job_id, []).remove(builder_id)

This occurs when a list is empty

>>> x = []
>>> x.remove('a')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: list.remove(x): x not in list

presumably because the two webhooks have put the global variable tracking builders build_counter into an inconsistent state.

It seems like the message should be marked as received, even if it is not handled correctly (move lie 371 before 363 ?) and that somewhere in https://github.com/Bioconductor/issue_tracker_github/blob/master/core.rb#L486-L507 duplicate webhooks should not be allowed to run (e.g., by adding a field to the data base that indicates a build is already in progress?)