-
Notifications
You must be signed in to change notification settings - Fork 451
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
db: make DB.Ingest return bytes ingested into L0 #1600
Comments
sumeerbhola
added a commit
to sumeerbhola/pebble
that referenced
this issue
Apr 19, 2022
To allow for async ingestion (being implemented) we don't reuse TableIngestInfo, and allow an approximate value for the bytes ingested into L0. These will be used in admission control to more precisely account for how much load an operation is adding to L0 (prototype is in https://github.com/cockroachdb/cockroach/blob/ec4149e310f9b383f87cc7ff8776258d72927394/pkg/util/admission/work_queue.go#L911). Fixes cockroachdb#1600
sumeerbhola
added a commit
that referenced
this issue
Apr 19, 2022
To allow for async ingestion (being implemented) we don't reuse TableIngestInfo, and allow an approximate value for the bytes ingested into L0. These will be used in admission control to more precisely account for how much load an operation is adding to L0 (prototype is in https://github.com/cockroachdb/cockroach/blob/ec4149e310f9b383f87cc7ff8776258d72927394/pkg/util/admission/work_queue.go#L911). Fixes #1600
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is needed for improvements to admission control that will use byte based tokens, that is part of cockroachdb/cockroach#75066 (see the prototype code in https://github.com/cockroachdb/cockroach/blob/ec4149e310f9b383f87cc7ff8776258d72927394/pkg/util/admission/work_queue.go#L911).
This is complicated by #25, since the actual placement into a Version will be asynchronously done. However, it is viable to produce an estimate at the time of Ingest based on the current spans of L0 files and memtables. This may be an over-estimate since (a) a memtable can be split into multiple sstables when flushing, and an ingested sstable may be able to slip through a gap, (b) an overlapping L0 file may get compacted down to Lbase by the time the ingest is applied to produce a new Version. An estimate is acceptable for admission control.
The text was updated successfully, but these errors were encountered: