Fix English and Spanish event pairing in Metro scraper #304
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Connects Metro-Records/la-metro-councilmatic#393 and opencivicdata/python-legistar-scraper#100.
The Metro scraper pairs English and Spanish Legistar events. When we are conducting a partial scrape, it's possible that one language event will be included, but not the other. We have logic to handle this case, however it relies on
since_datetime
being passed as a keyword argument to check whether a scrape is partial. This check did not work because we pass this value as a positional argument in thepython-legistar
library, hence the scraper never recognized partial scrapes and accordingly never looked for missing partners.This PR:
since_datetime
as a keyword argument toevents
in thescrape
method. This isn't technically necessary, but I thought it would be nice for consistency. The actual repair is made in Pass since_datetime as a keyword argument python-legistar-scraper#100, i.e., these PRs should be merged and deployed together.Test instructions
patch/hec/pass-kwarg
branch inpython-legistar-scraper
.python-legistar
:pip install -e path/to/python-legistar
.pupa update lametro events window=0.25 --fastmode --rpm=0