Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

db: Add IngestShared test to exercise shared ingestion #2574

Merged
merged 1 commit into from
Jun 20, 2023

Conversation

itsbilal
Copy link
Member

@itsbilal itsbilal commented May 26, 2023

This is a test-oriented change that adds more end-to-end testing for
ingesting shared sstables and also reading keys from them.

@itsbilal itsbilal requested a review from a team May 26, 2023 19:39
@itsbilal itsbilal self-assigned this May 26, 2023
@itsbilal itsbilal requested a review from RaduBerinde May 26, 2023 19:40
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@itsbilal itsbilal force-pushed the ingest-shared-test branch 3 times, most recently from 9dea30a to a87fad4 Compare May 29, 2023 18:17
@itsbilal itsbilal force-pushed the ingest-shared-test branch 2 times, most recently from bbf10b5 to c8d559f Compare June 12, 2023 16:29
@itsbilal
Copy link
Member Author

Rebased this now that #2538 has merged.

Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 1 of 16 files at r2, all commit messages.
Reviewable status: 1 of 16 files reviewed, 1 unresolved discussion (waiting on @itsbilal and @RaduBerinde)


testdata/ingest_shared line 482 at r2 (raw file):

  000004:[a@3#10,SET-c@9#10,SET]
5:
  000005:[b#2,RANGEDEL-d#2,RANGEDEL]

worth a test that exercises truncating the rangedel to the ingest sst bounds, ensuring it doesn't delete keys that fall within the physical sst bounds.

This is a test-oriented change that adds more end-to-end testing for
ingesting shared sstables and also reading keys from them.
Copy link
Member Author

@itsbilal itsbilal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR! Rebased to work with the newest changes.

Reviewable status: 0 of 16 files reviewed, 1 unresolved discussion (waiting on @jbowens and @RaduBerinde)


testdata/ingest_shared line 482 at r2 (raw file):

Previously, jbowens (Jackson Owens) wrote…

worth a test that exercises truncating the rangedel to the ingest sst bounds, ensuring it doesn't delete keys that fall within the physical sst bounds.

We actually don't support that, as we force virtual sstable bounds to fall in places where there are no range keys/deletes:

panic("pebble: bounds should not be truncated")

But on closer thought, we might need to revisit this; seeing as we could defragment [a,b) and [b,c) as [a,c) even if b is a crdb range boundary and we'd need to be able to create virtual sstable bounds at b.

@itsbilal itsbilal merged commit 5e3de5e into cockroachdb:master Jun 20, 2023
@itsbilal itsbilal deleted the ingest-shared-test branch June 20, 2023 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants