Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SstMetaData's size is 0 after flush #321

Closed
jiacai2050 opened this issue Oct 19, 2022 · 1 comment
Closed

SstMetaData's size is 0 after flush #321

jiacai2050 opened this issue Oct 19, 2022 · 1 comment
Labels
bug Something isn't working

Comments

@jiacai2050
Copy link
Contributor

jiacai2050 commented Oct 19, 2022

Describe this problem

When test #312, I find size of SstMetaData is 0 after flush memtable, this cause following error when restart(reading sst)

2022-10-19 11:49:32.342 ERRO [common_util/src/panic.rs:42] thread 'main' panicked 'Failed to create catalog manager: VisitSysCatalog { source: ReadStream { source: ErrWithSource { msg: "Read record batch", source: ReadFromSubIter { source: PullRecordBat
ch { source: DecodeRecordBatch { source: ExternalError(ParquetError(General("CachableParquetFileReader::get_metadata error: Execution error: file size of 0 is less than footer"))) } } } } } }' at "src/setup.rs:187"
   0: common_util::panic::set_panic_hook::{{closure}}

Steps to reproduce

  1. Insert some data
CREATE TABLE `cpu2` (
`my_ts` timestamp NOT NULL, 
 `t1` string tag,  
 `t2` string tag,  
 `value` double,
  TIMESTAMP KEY(my_ts)) 
  with (
enable_ttl = 'false'
);

insert into cpu2  (my_ts, t1,t2, `value`) 
values
(123, 'web', 'hz', 100),
(124, 'web','hz', 101),
(123, 'ios', 'sh', 100),
(124, 'ios', 'sh', 101);
  1. flush
curl --location --request POST 'localhost:5440/flush_memtable'
  1. restart service

Expected behavior

No panic

Additional Information

memtable is write to sst in
https://github.com/CeresDB/ceresdb/blob/c557007e6c56390d12debe2d69478dcd73ae4796/analytic_engine/src/instance/flush_compaction.rs#L641

but we update size after this, in https://github.com/CeresDB/ceresdb/blob/c557007e6c56390d12debe2d69478dcd73ae4796/analytic_engine/src/instance/flush_compaction.rs#L657
which is too late, since it's already persisted.

@jiacai2050
Copy link
Contributor Author

Fixed in #513

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant