storage: Cleanup states for dropped TABLEs/MVs

[hzxa21]

We already support DROP TABLE and DROP MATERIALIZED VIEW in frontent and meta but the data belonging to dropped tables/mvs still remain on storage occupying storage spaces and participating in compaction.

We should support state cleanup for dropped TABLEs/MVs. One feasible solution is to delete states via a compaction filter.

Related issues:
#191
#201
#1153
#1212
#1272

[neverchanje]

I used to think that we rely on background compaction (vacuum) to remove the data because the table is basically a prefix on large shared storage, right? So we can not drop the files immediately after dropping the table. So the question is, will the compaction processes remove the data with zero reference count?

[hzxa21]

I used to think that we rely on background compaction (vacuum) to remove the data because the table is basically a prefix on large shared storage, right? So we can not drop the files immediately after dropping the table.

Right. Data of dropped table/mv should be invisible to user immeidately but should be cleaned up asynchorously via background compaction.

[twocode]

@mikechesterwang This looks like to be one of your requirements.

[TennyZhuang]

BTW different MVs may share the same arrangement, so the real drop happens on the arrangement.

[skyzh]

Streaming and Meta part:

• Change all node to use table id as prefix, remove executor and shared prefix streaming: use table_id as topN prefix #2925
• Record table id belonging to a MV on meta (optional)
• Record drop prefix on drop MV / drop table / drop index

Storage part:

• Implement drop prefix compaction filter

[skyzh]

cc @Li0k may help change all stateful executors to use table id, and record all tables in one MV, before proceeding to implement compaction filter.

[hzxa21]

#3038 resolves this issue.