LQ scan / filtering simplification to speed up writes / Singer loads.#464

Merged

mildbyte merged 7 commits intomasterfrom bugfix/object-scanning-deletion

May 18, 2021

+229-183

Commits on May 14, 2021

When figuring out which objects to scan through, also add objects whose PKs overlap the PKs of the objects we are going to scan through and that come after them.
mildbyte
committed
Simplify and speed up the Singer fragment loading by not scanning the full current table when appending a new fragment to generate a changeset (to find out the old values of the rows we deleted).
mildbyte
committed
Change a Socrata smoke test dataset (data.healthcare.gov is currently down)
mildbyte
committed
Make the sgr cloud dump test deterministic.
mildbyte
committed
Singer taps output a sequence of intermediary bookmark states -- only write out the latest one when ingestion ends.
mildbyte
committed
Kind of fix the Singer tap-mysql test (not clear who should be doing the right thing here)
mildbyte
committed

Commits on May 18, 2021

Bump dependencies.
mildbyte
committed