Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add range for compaction filter context #192

Merged
merged 4 commits into from
Nov 6, 2020

Conversation

hicqu
Copy link

@hicqu hicqu commented Sep 16, 2020

Signed-off-by: qupeng qupeng@pingcap.com

This PR adds some information into CompactionFilter::Context:

  • start_key, end_key, is_end_key_inclusive: indicates the range of the compaction
  • file_numbers: file numbers of all involved SST files
  • table_properties: table properties of all involved SST files

These fields can be used in CompactionFilterFactory::create_compaction_filter.

Signed-off-by: qupeng <qupeng@pingcap.com>
@hicqu hicqu force-pushed the compaction-filter-range branch 2 times, most recently from f57abd2 to 3aeca4d Compare November 3, 2020 10:13
Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: qupeng <qupeng@pingcap.com>
Copy link
Collaborator

@yiwu-arbug yiwu-arbug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: qupeng <qupeng@pingcap.com>
Copy link

@Little-Wallace Little-Wallace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hicqu
Copy link
Author

hicqu commented Nov 6, 2020

/merge

@Connor1996 Connor1996 merged commit 0a4d235 into tikv:6.4.tikv Nov 6, 2020
@hicqu hicqu deleted the compaction-filter-range branch November 6, 2020 08:37
@@ -56,6 +59,18 @@ class CompactionFilter {
bool is_manual_compaction;
// Whether output files are in bottommost level or not.
bool is_bottommost_level;

// The range of the compaction.
Slice start_key;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these use the V2 struct above?

yiwu-arbug pushed a commit to tikv/rust-rocksdb that referenced this pull request Nov 9, 2020
update rocksdb to include tikv/rocksdb#192 for master

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@tabokie tabokie mentioned this pull request May 9, 2022
39 tasks
tabokie pushed a commit to tabokie/rocksdb that referenced this pull request May 12, 2022
* add range for compaction filter context

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
tabokie pushed a commit that referenced this pull request May 12, 2022
* add range for compaction filter context

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
v01dstar pushed a commit to v01dstar/rocksdb that referenced this pull request Feb 21, 2024
* add range for compaction filter context

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
v01dstar pushed a commit to v01dstar/rocksdb that referenced this pull request Feb 22, 2024
compaction_filter: add bottommost_level into context (tikv#160)

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>

add range for compaction filter context (tikv#192)

* add range for compaction filter context

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>

allow no_io for VersionSet::GetTableProperties (tikv#211)

* allow no_io for VersionSet::GetTableProperties

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>

expose seqno from compaction filter and iterator (tikv#215)

This PR supports to access `seqno` for every key/value pairs in compaction filter or iterator.
It's helpful to enhance GC in compaction filter in TiKV.

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>

allow to query DB stall status (tikv#226)

This PR adds a new property is-write-stalled to query whether the column family is in write stall or not.

In TiKV there is a compaction filter used for GC, in which DB::write is called. So if we can query whether the DB instance is stalled or not, we can skip to create more compaction filter instances to save some resources.

Signed-off-by: qupeng <qupeng@pingcap.com>
Signed-off-by: tabokie <xy.tao@outlook.com>

Fix compatibilty issue with Titan

Signed-off-by: v01dstar <yang.zhang@pingcap.com>

filter deletion in compaction filter (tikv#344)

And delay the buffer initialization of writable file to first actual write.

---------

Signed-off-by: tabokie <xy.tao@outlook.com>

Adjustments for compaptibilty with 8.10.facebook

Signed-off-by: v01dstar <yang.zhang@pingcap.com>

Adjust tikv related changes with upstream

Signed-off-by: v01dstar <yang.zhang@pingcap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants