Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crash_test: memtablerep is incorrectly passed to db_stress #93

Closed
isaac-io opened this issue Aug 1, 2022 · 0 comments · Fixed by #94
Closed

crash_test: memtablerep is incorrectly passed to db_stress #93

isaac-io opened this issue Aug 1, 2022 · 0 comments · Fixed by #94
Assignees
Labels
bug Something isn't working

Comments

@isaac-io
Copy link
Contributor

isaac-io commented Aug 1, 2022

After the changes in #90, running crash test fails wit the following output:

[2022-07-31 21:14:00.517032] Running db_stress with pid=37005: ./db_stress --acquire_snapshot_one_in=10000 --adaptive_readahead=1 --allow_concurrent_memtable_write=1 --async_io=0 --avoid_flush_during_recovery=0 -
-avoid_unnecessary_blocking_io=1 --backup_max_size=104857600 --backup_one_in=100000 --batch_protection_bytes_per_key=0 --block_size=4096 --bloom_bits=3 --bottommost_compression_type=lz4 --cache_index_and_filter_b
locks=0 --cache_size=8388608 --checkpoint_one_in=0 --checksum_type=kXXH3 --clear_column_family_one_in=0 --compact_files_one_in=1000000 --compact_range_one_in=1000000 --compaction_ttl=0 --compare_full_db_state_sna
pshot=1 --compression_max_dict_buffer_bytes=0 --compression_max_dict_bytes=0 --compression_parallel_threads=1 --compression_type=zstd --compression_zstd_max_train_bytes=0 --continuous_verification_interval=0 --cu
stomopspercent=0 --data_block_hash_table_util_ratio=0.33 --data_block_index_type=0 --db=/tmp/rocksdb_crashtest_whiteboxywygu9an --db_write_buffer_size=0 --delpercent=11 --delrangepercent=0 --destroy_db_initially=
0 --detect_filter_construct_corruption=1 --disable_wal=0 --enable_compaction_filter=0 --enable_pipelined_write=0 --expected_values_dir=/tmp/rocksdb_crashtest_expected_kl069o3q --fail_if_options_file_error=0 --fil
e_checksum_impl=big --flush_one_in=1000000 --format_version=4 --get_current_wal_file_one_in=0 --get_live_files_one_in=100000 --get_property_one_in=1000000 --get_sorted_wal_files_one_in=0 --index_block_restart_int
erval=4 --index_type=0 --iterpercent=55 --key_len_percent_dist=4,29,5,19,24,19 --kill_exclude_prefixes=WritableFileWriter::Append,WritableFileWriter::WriteBuffered --kill_random_test=88889 --level_compaction_dyna
mic_level_bytes=True --long_running_snapshots=1 --mark_for_compaction_one_file_in=10 --max_background_compactions=20 --max_bytes_for_level_base=10485760 --max_key=10485760 --max_key_len=6 --max_manifest_file_size
=1073741824 --max_write_batch_group_size_bytes=16777216 --max_write_buffer_number=3 --max_write_buffer_size_to_maintain=1048576 --memtable_prefix_bloom_size_ratio=0.001 --memtable_whole_key_filtering=0 --memtable
rep=('speedb.HashSpdRepFactory',) --mmap_read=1 --mock_direct_io=False --nooverwritepercent=0 --num_iterations=14 --open_files=100 --open_metadata_write_fault_one_in=0 --open_read_fault_one_in=32 --open_write_fau
lt_one_in=16 --ops_per_thread=20000000 --optimize_filters_for_memory=1 --paranoid_file_checks=1 --partition_filters=0 --partition_pinning=0 --pause_background_one_in=1000000 --periodic_compaction_seconds=0 --pref
ix_size=8 --prefixpercent=1 --prepopulate_block_cache=0 --progress_reports=0 --read_fault_one_in=1000 --readpercent=7 --recycle_log_file_num=0 --reopen=20 --reserve_table_reader_memory=0 --ribbon_starting_level=9
99 --secondary_cache_fault_one_in=0 --seed=1544212072 --snapshot_hold_ops=100000 --sst_file_manager_bytes_per_sec=104857600 --sst_file_manager_bytes_per_truncate=0 --subcompactions=1 --sync=False --sync_fault_inj
ection=False --sync_wal_one_in=100000 --target_file_size_base=2097152 --target_file_size_multiplier=2 --test_batches_snapshots=0 --top_level_index_pinning=0 --txn_write_policy=1 --unordered_write=1 --unpartitione
d_pinning=1 --use_block_based_filter=0 --use_clock_cache=0 --use_direct_io_for_flush_and_compaction=0 --use_direct_reads=0 --use_full_merge_v1=False --use_merge=0 --use_multiget=1 --use_txn=1 --user_timestamp_siz
e=0 --value_size_mult=32 --verify_before_write=False --verify_checksum=1 --verify_checksum_one_in=1000000 --verify_db_one_in=100000 --wal_compression=none --write_buffer_size=1073741824 --write_dbid_to_manifest=0
 --writepercent=26

[2022-07-31 21:14:00.525299] WARNING: db_stress (pid=37005) ended before kill: exitcode=1

check_mode=0, kill option=88889, exitcode=1

2022/07/31-21:14:00  Initializing db_stress
RocksDB version           : 7.2
Format version            : 4
TransactionDB             : true
Two write queues:         : false
Write policy              : 1
Snapshot cache bits       : 7
Commit cache bits         : 23
last cwb for recovery    : false
Stacked BlobDB            : false
Read only mode            : false
Atomic flush              : false
Column families           : 10
Clear CFs one in          : 0
Number of threads         : 32
Ops per thread            : 20000000
Time to live(sec)         : unused
Read percentage           : 7%
Prefix percentage         : 1%
Write percentage          : 26%
Delete percentage         : 11%
Delete range percentage   : 0%
No overwrite percentage   : 0%
Iterate percentage        : 55%
Custom ops percentage     : 0%
DB-write-buffer-size      : 0
Write-buffer-size         : 1073741824
Iterations                : 14
Max key                   : 10485760
Ratio #ops/#keys          : 61.035156
Num times DB reopens      : 20
Batches/snapshots         : 0
Do update in place        : 0
Num keys per lock         : 4
Compression               : ZSTD
Bottommost Compression    : LZ4
Checksum type             : kXXH3
File checksum impl        : big
Bloom bits / key          : 3.000000
Max subcompactions        : 1
Use MultiGet              : true
Memtablerep               : ('speedb.HashSpdRepFactory',)
Test kill odd             : 88889
Skipping kill points prefixes:
  WritableFileWriter::Append
  WritableFileWriter::WriteBuffered
Periodic Compaction Secs  : 0
Compaction TTL            : 0
Background Purge          : 1
Write DB ID to manifest   : 0
Max Write Batch Group Size: 16777216
Use dynamic level         : 1
Read fault one in         : 1000
Write fault one in        : 0
Open metadata write fault one in:
                            0
Sync fault injection      : 0
Best efforts recovery     : 0
Fail if OPTIONS file error: 0
User timestamp size bytes : 0
WAL compression           : none
data_block_index_type : 0
data_block_hash_table_util_ratio : 0.330000
------------------------------------------------

MemTableFactory creation failed: Not implemented: Could not load MemTableRepFactory: ('speedb.HashSpdRepFactory',)

TEST FAILED. See kill option and exit code above!!!

The reason is a bug on my part when I added the choice between the skip list memtable and the Speed one and accidentally added a comma to the end of the line, which was later formatted by black with a proper tuple construction.

To Reproduce

make crash_test

Expected behavior

The test doesn't crash because of the choice of memtable.

System (please complete the following information)

  • OS: Ubuntu 20.04
  • Compiler and version: GCC 9.4.0

Additional context

N/A

@isaac-io isaac-io added the bug Something isn't working label Aug 1, 2022
@isaac-io isaac-io self-assigned this Aug 1, 2022
isaac-io added a commit that referenced this issue Aug 1, 2022
As part of #90, I accidentally added a comma at the end of the
line when choosing the memtable reperesntation to use. This is
a tuple construction syntax and was formatted as such by Black,
but unfortunately missed by me and during review.

Fix it by removing the comma so that the argument isn't passed
to db_stress as a Python tuple.
isaac-io added a commit that referenced this issue Aug 1, 2022
As part of #90, I accidentally added a comma at the end of the
line when choosing the memtable reperesntation to use. This is
a tuple construction syntax and was formatted as such by Black,
but unfortunately missed by me and during review.

Fix it by removing the comma so that the argument isn't passed
to db_stress as a Python tuple.
isaac-io added a commit that referenced this issue Aug 1, 2022
…h test

The test seems to fail for now, so in order to avoid headaches for devs
working on `main`, let's not test the new memtable until the issues are
fixed.
isaac-io added a commit that referenced this issue Oct 19, 2022
As part of #90, I accidentally added a comma at the end of the
line when choosing the memtable reperesntation to use. This is
a tuple construction syntax and was formatted as such by Black,
but unfortunately missed by me and during review.

Fix it by removing the comma so that the argument isn't passed
to db_stress as a Python tuple.
isaac-io added a commit that referenced this issue Oct 19, 2022
As part of #90, I accidentally added a comma at the end of the
line when choosing the memtable reperesntation to use. This is
a tuple construction syntax and was formatted as such by Black,
but unfortunately missed by me and during review.

Fix it by removing the comma so that the argument isn't passed
to db_stress as a Python tuple.
isaac-io added a commit that referenced this issue Oct 19, 2022
As part of #90, I accidentally added a comma at the end of the
line when choosing the memtable reperesntation to use. This is
a tuple construction syntax and was formatted as such by Black,
but unfortunately missed by me and during review.

Fix it by removing the comma so that the argument isn't passed
to db_stress as a Python tuple.
Yuval-Ariel pushed a commit that referenced this issue Nov 23, 2022
As part of #90, I accidentally added a comma at the end of the
line when choosing the memtable reperesntation to use. This is
a tuple construction syntax and was formatted as such by Black,
but unfortunately missed by me and during review.

Fix it by removing the comma so that the argument isn't passed
to db_stress as a Python tuple.
Yuval-Ariel pushed a commit that referenced this issue Nov 25, 2022
As part of #90, I accidentally added a comma at the end of the
line when choosing the memtable reperesntation to use. This is
a tuple construction syntax and was formatted as such by Black,
but unfortunately missed by me and during review.

Fix it by removing the comma so that the argument isn't passed
to db_stress as a Python tuple.
Yuval-Ariel pushed a commit that referenced this issue Apr 30, 2023
As part of #90, I accidentally added a comma at the end of the
line when choosing the memtable reperesntation to use. This is
a tuple construction syntax and was formatted as such by Black,
but unfortunately missed by me and during review.

Fix it by removing the comma so that the argument isn't passed
to db_stress as a Python tuple.
udi-speedb pushed a commit that referenced this issue Oct 31, 2023
As part of #90, I accidentally added a comma at the end of the
line when choosing the memtable reperesntation to use. This is
a tuple construction syntax and was formatted as such by Black,
but unfortunately missed by me and during review.

Fix it by removing the comma so that the argument isn't passed
to db_stress as a Python tuple.
udi-speedb pushed a commit that referenced this issue Dec 1, 2023
As part of #90, I accidentally added a comma at the end of the
line when choosing the memtable reperesntation to use. This is
a tuple construction syntax and was formatted as such by Black,
but unfortunately missed by me and during review.

Fix it by removing the comma so that the argument isn't passed
to db_stress as a Python tuple.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant