Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sharded BlueStore - 3 - resharding tool #29087

Closed
wants to merge 36 commits into from
Closed

Sharded BlueStore - 3 - resharding tool #29087

wants to merge 36 commits into from

Commits on Jan 16, 2020

  1. common/KeyValueDB: Now IteratorImpl and WholeSpaceIteratorImpl share … 

    …useful interfaces.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    427401d View commit details
    Browse the repository at this point in the history

  2. common/KeyValueDB: Refactor inheritance to expose bidirectonal iterator. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    68778b6 View commit details
    Browse the repository at this point in the history

  3. common/KeyValueDB: Modification of column families. Implementation fo… 

    …r RocksDB.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    fc9d0e1 View commit details
    Browse the repository at this point in the history

  4. common/KeyValueDB: Created type ColumnFamilyHandle. Refactor Iterators. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    765642e View commit details
    Browse the repository at this point in the history

  5. common/KeyValueDB: Added merge operator that handles all prefixes. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    6f75ffc View commit details
    Browse the repository at this point in the history

  6. common/KeyValueDB: Fixed debug class for printing RocksDB write_batch… 

    … transactions.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    bc958d7 View commit details
    Browse the repository at this point in the history

  7. common/KeyValueDB: Column families related: refactor, fix doc, move c… 

    …ode around, add r/w locking.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    0c3ed09 View commit details
    Browse the repository at this point in the history

  8. common/KeyValueDB: Fix iterator on column family for rocksdb. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    130aa0c View commit details
    Browse the repository at this point in the history

  9. common/KeyValueDB: Add proper compaction contol for column families. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    2301e0b View commit details
    Browse the repository at this point in the history

  10. common/KeyValueDB: Fixed deletion of column family. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    8ba1607 View commit details
    Browse the repository at this point in the history

  11. common/KeyValueDB: Fix closing handle for 'default_cf'. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    ccdc1c9 View commit details
    Browse the repository at this point in the history

  12. common/KeyValueDB: Now column_family_handle no longer returns default… 

    …_cf for "".

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    fdf30c1 View commit details
    Browse the repository at this point in the history

  13. common/KeyValueDB: Fixes and improvements, locks when using cf_get_{m… 

    …ono_}handle()

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    5686f58 View commit details
    Browse the repository at this point in the history

  14. common/KeyValueDB: Fix rm_range_keys for column family cases 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    22a1031 View commit details
    Browse the repository at this point in the history

  15. common/KeyValueDB: Fix delete_column_family 

    Inverted condition occurred.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    d095479 View commit details
    Browse the repository at this point in the history

  16. common/KeyValueDB: Fix tests for estimate column size 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    8caacc8 View commit details
    Browse the repository at this point in the history

  17. BlueStore/sharding: Test-grade work on using RocksDb in sharded mode. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    1551703 View commit details
    Browse the repository at this point in the history

  18. BlueStore/sharding: Test enable sharing of RocksDB 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    68a0c1c View commit details
    Browse the repository at this point in the history

  19. BlueStore/sharding: 0 shards means 'default' column family. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    8765c65 View commit details
    Browse the repository at this point in the history

  20. BlueStore/sharding: Added SinglePrefixIteratorMerged_Impl, merges sha… 

    …rds into one iterator.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    516d450 View commit details
    Browse the repository at this point in the history

  21. BlueStore/sharding: Added ShardedIteratorBase and refactored iterators. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    2e4b016 View commit details
    Browse the repository at this point in the history

  22. BlueStore/sharding: fixup, adaptation to changes to rockskvdb 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    5fc583e View commit details
    Browse the repository at this point in the history

  23. BlueStore/sharding: Now 'bluestore_rocksdb_cf' option allows to defin… 

    …e number of shards.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    d751256 View commit details
    Browse the repository at this point in the history

  24. BlueStore/sharding: Properly create BlueStore_DB_Hash.h 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    03f0377 View commit details
    Browse the repository at this point in the history

  25. BlueStore/sharding: Add function get_sharding that parses sharding de… 

    …finition.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    3094580 View commit details
    Browse the repository at this point in the history

  26. BlueStore/sharding: Added limited hashing for OMAP keys, only first 8… 

    … bytes are used. Now all OMAP for single object are in one shard.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    03c3764 View commit details
    Browse the repository at this point in the history

  27. BlueStore/sharding: add missing estmate_prefix_size implementation. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    effeacf View commit details
    Browse the repository at this point in the history

  28. BlueStore/sharding: bugfix, not mutable value does not switch storage 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    d2dd8a0 View commit details
    Browse the repository at this point in the history

  29. BlueStore/sharding: bugfix, do not delete column family handles that … 

    …were not completely created

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    d9daf5c View commit details
    Browse the repository at this point in the history

  30. common/tools: Added 'reshard' action to ceph-bluestore-tool 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    805d6d8 View commit details
    Browse the repository at this point in the history

  31. common/tools: Now reshard procedure releases iterator periodically to… 

    … release locked .sst tables.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    a26361b View commit details
    Browse the repository at this point in the history

  32. common/tools: Now ceph-kvstore-tool can fully operate on column famil… 

    …ies.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    b1a2740 View commit details
    Browse the repository at this point in the history

  33. common/tools: Fix cli tests for ceph-kvstore-tool 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    3c91eae View commit details
    Browse the repository at this point in the history

  34. common/tools: Now ceph-kvstore-tool can 'reshard' 'b' columns. 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    3e2084c View commit details
    Browse the repository at this point in the history

  35. common/tools: fix, now resharing properly opens bluestore 

    Now fully mounting bluestore, to allow to provide additional space block->block.db

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    b1173af View commit details
    Browse the repository at this point in the history

  36. common/tools: Added 'list-columns' option to ceph-kvstore-tool 

    Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
    @aclamk
    aclamk committed Jan 16, 2020
    Configuration menu
    Copy the full SHA
    fe8efbc View commit details
    Browse the repository at this point in the history