Introduce file inspector and add FileSystemInspectedEnv #216
Conversation
Signed-off-by: tabokie <xy.tao@outlook.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
include/rocksdb/env.h
Outdated
| @@ -364,6 +364,8 @@ class Env { | |||
| // Priority for requesting bytes in rate limiter scheduler | |||
| enum IOPriority { IO_LOW = 0, IO_HIGH = 1, IO_TOTAL = 2 }; | |||
|
|
|||
| enum IOType { IO_UNCATEGORIZED = 0, IO_FLUSH = 1, IO_COMPACTION = 2 }; | |||
There was a problem hiding this comment.
I want to avoid adding IO type to rocksdb code. I think the rust FileSystemInpector can obtain the thread-local IO type by itself. To tell flush and compaction apart, we can set the thread-local IO type in the OnFlushBegin/OnCompactionBegin callbacks.
There was a problem hiding this comment.
Sounds great, save lots of trouble.
There was a problem hiding this comment.
@yiwu-arbug But I think OnCompactionBegin doesn't respect subcompaction.
There was a problem hiding this comment.
sad. how about adding an sub-compaction begin event?
There was a problem hiding this comment.
Also OnFlushComplete is tricky, it can run on another thread. But maybe we just make use of both of the Begin event.
There was a problem hiding this comment.
It's doable, I'll leave it for followup PRs.
Signed-off-by: tabokie <xy.tao@outlook.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
Signed-off-by: tabokie <xy.tao@outlook.com>
|
Comments addressed. @yiwu-arbug @Connor1996 |
Signed-off-by: tabokie <xy.tao@outlook.com>
Introduce `FileSystemInspector` interface and `FileSystemInspectedEnv`. With `FileSystemInspectedEnv`, each file operation must first consult `FileSystemInspector` before reaching actual IO logic. Signed-off-by: tabokie <xy.tao@outlook.com>
Introduce `FileSystemInspector` interface and `FileSystemInspectedEnv`. With `FileSystemInspectedEnv`, each file operation must first consult `FileSystemInspector` before reaching actual IO logic. Signed-off-by: tabokie <xy.tao@outlook.com>
Introduce
FileSystemInspectorabstract interface andFileSystemInspectedEnv. WithFileSystemInspectedEnv, each file operation must first consultFileSystemInspectorbefore reaching actual IO logic.Also, addIOTypesupport toSequentialFile. Before compaction or flush, the specific type can be passed to inspector through newly added methods. Notice it might be possible to merge existingTableReaderCallerwith IOType, maybe as a followup task.