Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor BatchDbWriter to fix bugs #4176

Merged
merged 3 commits into from
Aug 7, 2024
Merged

refactor BatchDbWriter to fix bugs #4176

merged 3 commits into from
Aug 7, 2024

Conversation

simonjiao
Copy link
Collaborator

@simonjiao simonjiao commented Aug 7, 2024

  1. fix a typo
  2. remove unused codes

Pull request type

Please check the type of change your PR introduces:

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Documentation content changes
  • Other (please describe):

What is the current behavior?

Issue Number: N/A

What is the new behavior?

Other information

Summary by CodeRabbit

  • New Features

    • Simplified initialization of BlockDAG instances by removing the new_by_config function, streamlining the process for users.
    • Enhanced functionality of BatchDbWriter with direct database access for improved data handling during write operations.
  • Documentation

    • Updated the visibility of the get_cf_handle function to public, allowing broader access for other modules.
  • Chores

    • Minor adjustments to import statements and internal structuring for improved clarity and effectiveness.

1. fix a typo
2. remove unused codes
Copy link

coderabbitai bot commented Aug 7, 2024

Walkthrough

The recent changes enhance the BlockDAG and database handling in the flexidag project. Key modifications include the removal of the new_by_config function, the introduction of database references to improve integration, and broader accessibility to database handles. These updates streamline the interface and enhance database interactions, simplifying usage while promoting better performance and data integrity across the codebase.

Changes

Files Change Summary
flexidag/src/blockdag.rs Removed the new_by_config function from BlockDAG, simplifying instance initialization.
flexidag/src/consensusdb/writer.rs Added a db field to BatchDbWriter, enhancing its functionality with direct database access. Constructor updated to require a DBStorage reference.
flexidag/src/consensusdb/* Updated various implementations in BatchDbWriter to improve database interactions during writes.
storage/src/db_storage/mod.rs Changed get_cf_handle from private to public, broadening access to retrieve column family handles.
flexidag/tests/tests.rs Corrected a typographical error in the test function name.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant BlockDAG
    participant DBStorage
    participant BatchDbWriter

    User->>BlockDAG: Create instance
    BlockDAG->>DBStorage: Initialize with default config
    BlockDAG->>BatchDbWriter: Create BatchDbWriter with DB reference
    BatchDbWriter->>DBStorage: Access database for write operations
    BatchDbWriter-->>User: Operations complete
Loading

🐇 "In the land of hops and plays,
Changes bloom like sunny rays.
With a skip and a joyful cheer,
New paths shine and troubles clear.
Let's twirl and dance, oh what a sight,
For code has changed, and all feels right!" 🌼


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 10da414 and 7859e7d.

Files selected for processing (9)
  • flexidag/src/blockdag.rs (3 hunks)
  • flexidag/src/consensusdb/consensus_ghostdag.rs (1 hunks)
  • flexidag/src/consensusdb/consensus_header.rs (1 hunks)
  • flexidag/src/consensusdb/consensus_reachability.rs (2 hunks)
  • flexidag/src/consensusdb/consensus_relations.rs (2 hunks)
  • flexidag/src/consensusdb/db.rs (1 hunks)
  • flexidag/src/consensusdb/writer.rs (1 hunks)
  • flexidag/tests/tests.rs (1 hunks)
  • storage/src/db_storage/mod.rs (1 hunks)
Files skipped from review due to trivial changes (1)
  • flexidag/tests/tests.rs
Additional comments not posted (12)
flexidag/src/consensusdb/writer.rs (3)

42-42: LGTM! The addition of the db field enhances functionality.

The new field db in BatchDbWriter allows direct access to the database storage, improving the robustness of data handling.


46-47: LGTM! The updated constructor enhances functionality.

The constructor BatchDbWriter::new now accepts a DBStorage reference, allowing better integration with the database.


55-57: LGTM! The updated methods enhance robustness.

The put and delete methods now use the db field to retrieve the column family handle, ensuring operations are performed on the correct column family.

Also applies to: 63-65

flexidag/src/consensusdb/consensus_header.rs (1)

128-128: LGTM! The updated instantiation enhances functionality.

The constructor for BatchDbWriter now includes an additional parameter &self.db, enabling better integration with the database.

Also applies to: 136-136

flexidag/src/consensusdb/consensus_relations.rs (3)

106-106: LGTM! Ensures correct database context.

The change ensures that BatchDbWriter has access to the necessary database context during its operations.


110-110: LGTM! Ensures correct database context.

The change ensures that BatchDbWriter has access to the necessary database context during its operations.


120-120: LGTM! Ensures correct database context.

The change ensures that BatchDbWriter has access to the necessary database context during its operations.

flexidag/src/blockdag.rs (1)

16-16: LGTM! Simplifies initialization process.

The removal of these imports indicates that the new_by_config function, which relied on these imports, has been removed. This simplifies the initialization process of BlockDAG.

storage/src/db_storage/mod.rs (1)

193-193: LGTM! Broadens accessibility.

Making this function public allows external access to retrieve the handle for a specific column family from the RocksDB database. Ensure that this change does not introduce any unintended side effects.

Verification successful

Verify external usages of get_cf_handle

The function get_cf_handle is used extensively within the storage/src/db_storage/mod.rs file and also in flexidag/src/consensusdb/writer.rs. We need to ensure that there are no other external usages that could introduce unintended side effects.


Verified: No unintended side effects

The function get_cf_handle is used internally within storage/src/db_storage/mod.rs and externally in flexidag/src/consensusdb/writer.rs. No other external usages were found, indicating that making the function public does not introduce any unintended side effects.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that making `get_cf_handle` public does not introduce any unintended side effects.

# Test: Search for the usage of `get_cf_handle`. Expect: No unintended side effects.
rg --type rust -A 5 $'get_cf_handle'

Length of output: 5992


Script:

#!/bin/bash
# Description: Verify that making `get_cf_handle` public does not introduce any unintended side effects.

# Test: Search for the usage of `get_cf_handle` in the entire codebase to ensure no unintended side effects.
rg --type rust 'get_cf_handle'

Length of output: 1633

flexidag/src/consensusdb/consensus_reachability.rs (2)

258-269: LGTM! Ensure consistency in BatchDbWriter usage.

The change to pass &self.db to BatchDbWriter is approved as it enhances database interaction. Verify that similar changes are consistently applied across the codebase where BatchDbWriter is instantiated.

Verification successful

Consistent usage of BatchDbWriter with &self.db verified.

The change to pass &self.db to BatchDbWriter has been consistently applied across the codebase, ensuring uniformity in database interactions.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify consistent usage of `BatchDbWriter` with `&self.db`.

# Test: Search for `BatchDbWriter` instantiation. Expect: Consistent usage of `&self.db`.
rg --type rust -A 3 $'BatchDbWriter::new'

Length of output: 3511


142-144: LGTM! Ensure consistency in BatchDbWriter usage.

The change to pass &self.db to BatchDbWriter is approved as it enhances database interaction. Verify that similar changes are consistently applied across the codebase where BatchDbWriter is instantiated.

Verification successful

Verified: Consistent usage of BatchDbWriter with &self.db.

The instantiation of BatchDbWriter with &self.db or &db has been consistently applied across the codebase, ensuring uniform database interaction.

  • flexidag/src/consensusdb/consensus_ghostdag.rs
  • flexidag/src/consensusdb/consensus_relations.rs
  • flexidag/src/consensusdb/consensus_header.rs
  • flexidag/src/consensusdb/consensus_reachability.rs
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify consistent usage of `BatchDbWriter` with `&self.db`.

# Test: Search for `BatchDbWriter` instantiation. Expect: Consistent usage of `&self.db`.
rg --type rust -A 3 $'BatchDbWriter::new'

Length of output: 3511

flexidag/src/consensusdb/consensus_ghostdag.rs (1)

234-236: LGTM! Ensure consistency in BatchDbWriter usage.

The change to pass &self.db to BatchDbWriter is approved as it enhances database interaction. Verify that similar changes are consistently applied across the codebase where BatchDbWriter is instantiated.

Verification successful

Verified: Consistent usage of BatchDbWriter with &self.db.

The instantiation of BatchDbWriter with &self.db has been consistently applied across the codebase, ensuring uniform database interaction.

  • flexidag/src/consensusdb/consensus_reachability.rs
  • flexidag/src/consensusdb/consensus_ghostdag.rs
  • flexidag/src/consensusdb/consensus_relations.rs
  • flexidag/src/consensusdb/consensus_header.rs
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify consistent usage of `BatchDbWriter` with `&self.db`.

# Test: Search for `BatchDbWriter` instantiation. Expect: Consistent usage of `&self.db`.
rg --type rust -A 3 $'BatchDbWriter::new'

Length of output: 3511

1. increase blockdag's default cache size to 1024 for test cases
Copy link

github-actions bot commented Aug 7, 2024

Benchmark for dc7676a

Click to view benchmark
Test Base PR %
accumulator_append 752.4±66.58µs 761.7±96.70µs +1.24%
block_apply/block_apply_10 365.3±7.67ms 363.7±22.54ms -0.44%
block_apply/block_apply_1000 38.9±0.86s 39.1±0.60s +0.51%
get_with_proof/db_store 46.4±5.27µs 50.5±7.31µs +8.84%
get_with_proof/mem_store 36.7±1.81µs 38.0±5.90µs +3.54%
put_and_commit/db_store/1 111.7±6.37µs 125.9±17.59µs +12.71%
put_and_commit/db_store/10 1015.4±87.22µs 1040.2±111.19µs +2.44%
put_and_commit/db_store/100 10.3±2.73ms 9.3±1.07ms -9.71%
put_and_commit/db_store/5 517.6±40.47µs 544.4±37.32µs +5.18%
put_and_commit/db_store/50 4.8±0.48ms 5.3±0.47ms +10.42%
put_and_commit/mem_store/1 71.2±6.85µs 71.6±6.10µs +0.56%
put_and_commit/mem_store/10 665.9±48.62µs 667.0±47.47µs +0.17%
put_and_commit/mem_store/100 6.5±0.33ms 6.6±0.44ms +1.54%
put_and_commit/mem_store/5 335.0±29.72µs 350.7±57.72µs +4.69%
put_and_commit/mem_store/50 3.2±0.15ms 3.2±0.19ms 0.00%
query_block/query_block_in(10)_times(100) 4.5±0.15ms 4.5±0.13ms 0.00%
query_block/query_block_in(10)_times(1000) 46.3±2.03ms 44.8±1.11ms -3.24%
query_block/query_block_in(10)_times(10000) 442.9±5.27ms 438.8±10.40ms -0.93%
query_block/query_block_in(1000)_times(100) 1175.7±25.12µs 1163.6±158.84µs -1.03%
query_block/query_block_in(1000)_times(1000) 11.7±0.22ms 10.9±0.14ms -6.84%
query_block/query_block_in(1000)_times(10000) 119.0±3.94ms 110.6±4.12ms -7.06%
storage_transaction 1102.9±412.87µs 988.8±350.82µs -10.35%
vm/transaction_execution/1 417.1±18.33ms 408.8±6.39ms -1.99%
vm/transaction_execution/10 129.6±2.77ms 129.7±6.42ms +0.08%
vm/transaction_execution/20 118.0±2.69ms 121.7±4.16ms +3.14%
vm/transaction_execution/5 162.9±6.83ms 159.0±5.56ms -2.39%
vm/transaction_execution/50 139.3±5.04ms 145.4±9.56ms +4.38%

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 7859e7d and 82e286d.

Files selected for processing (3)
  • flexidag/src/blockdag.rs (3 hunks)
  • flexidag/src/consensusdb/writer.rs (1 hunks)
  • storage/src/db_storage/mod.rs (12 hunks)
Files skipped from review as they are similar to previous changes (2)
  • flexidag/src/consensusdb/writer.rs
  • storage/src/db_storage/mod.rs
Additional comments not posted (4)
flexidag/src/blockdag.rs (4)

Line range hint 1-16:
LGTM! Imports cleanup.

The removal of unused imports (Path and RocksdbConfig) is appropriate and helps in keeping the code clean.


Line range hint 34-34:
LGTM! Removal of new_by_config function.

The removal of the new_by_config function simplifies the BlockDAG interface and promotes a more straightforward usage pattern.


63-67: Verify the usage of temp_dir.

Ensure that the temp_dir function from starcoin_config is correctly used and that it aligns with the overall refactoring goals.

Verification successful

The usage of temp_dir is correct.

The temp_dir function from starcoin_config is consistently used across the codebase for creating temporary directories, aligning with the overall refactoring goals.

  • The function is correctly invoked in the create_for_testing function in flexidag/src/blockdag.rs.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of `temp_dir` function in the codebase.

# Test: Search for the `temp_dir` function usage. Expect: Correct usage in the codebase.
rg --type rust -A 5 $'temp_dir'

Length of output: 28114


Line range hint 119-119:
Verify error handling in commit function.

Ensure that the error handling in the commit function is appropriate and that it aligns with the overall refactoring goals.

Verification successful

To verify the error handling in the commit function, let's review the complete implementation of the function in flexidag/src/blockdag.rs.


To fully verify the error handling in the commit function, we need to retrieve the remaining part of the function.


The error handling in the commit function is appropriate and aligns with the overall refactoring goals.

  • The function uses the ? operator to propagate errors.
  • Specific errors are handled with detailed logging and actions.
  • The use of bail! ensures that errors are propagated correctly.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the error handling in the `commit` function.

# Test: Search for the error handling in the `commit` function. Expect: Appropriate error handling.
rg --type rust -A 20 $'fn commit'

Length of output: 16762


Script:

#!/bin/bash
# Description: Retrieve the complete implementation of the `commit` function in `flexidag/src/blockdag.rs` to verify error handling.

# Search for the `commit` function in `flexidag/src/blockdag.rs` and display its full implementation.
rg --type rust -A 50 'pub fn commit' flexidag/src/blockdag.rs

Length of output: 2042


Script:

#!/bin/bash
# Description: Retrieve the complete implementation of the `commit` function in `flexidag/src/blockdag.rs` to verify error handling.

# Search for the `commit` function in `flexidag/src/blockdag.rs` and display its full implementation.
rg --type rust -A 100 'pub fn commit' flexidag/src/blockdag.rs

Length of output: 4379

@simonjiao simonjiao requested a review from nkysg August 7, 2024 10:35
Copy link

github-actions bot commented Aug 7, 2024

Benchmark for e575270

Click to view benchmark
Test Base PR %
accumulator_append 776.0±116.00µs 764.1±102.33µs -1.53%
block_apply/block_apply_10 377.5±17.56ms 361.4±5.08ms -4.26%
block_apply/block_apply_1000 38.7±1.22s 39.1±0.57s +1.03%
get_with_proof/db_store 53.2±13.23µs 57.6±16.67µs +8.27%
get_with_proof/mem_store 38.4±2.03µs 36.8±0.64µs -4.17%
put_and_commit/db_store/1 118.3±9.13µs 112.2±5.11µs -5.16%
put_and_commit/db_store/10 1096.8±148.11µs 1024.0±45.98µs -6.64%
put_and_commit/db_store/100 9.8±0.91ms 9.4±0.41ms -4.08%
put_and_commit/db_store/5 542.6±24.00µs 517.8±24.85µs -4.57%
put_and_commit/db_store/50 5.1±0.52ms 4.9±0.36ms -3.92%
put_and_commit/mem_store/1 71.1±6.17µs 71.9±7.04µs +1.13%
put_and_commit/mem_store/10 676.4±63.89µs 668.0±47.81µs -1.24%
put_and_commit/mem_store/100 6.7±0.75ms 6.4±0.31ms -4.48%
put_and_commit/mem_store/5 334.4±26.68µs 352.9±43.70µs +5.53%
put_and_commit/mem_store/50 3.3±0.23ms 3.2±0.19ms -3.03%
query_block/query_block_in(10)_times(100) 4.5±0.13ms 4.5±0.12ms 0.00%
query_block/query_block_in(10)_times(1000) 44.5±1.05ms 46.9±4.30ms +5.39%
query_block/query_block_in(10)_times(10000) 448.7±10.96ms 450.8±15.18ms +0.47%
query_block/query_block_in(1000)_times(100) 1187.2±16.54µs 1172.2±54.06µs -1.26%
query_block/query_block_in(1000)_times(1000) 12.0±0.22ms 11.6±0.48ms -3.33%
query_block/query_block_in(1000)_times(10000) 119.9±2.06ms 113.8±3.07ms -5.09%
storage_transaction 1142.9±465.86µs 1087.3±429.27µs -4.86%
vm/transaction_execution/1 447.7±35.38ms 422.5±21.35ms -5.63%
vm/transaction_execution/10 144.9±8.82ms 130.2±5.19ms -10.14%
vm/transaction_execution/20 123.6±6.26ms 124.9±5.98ms +1.05%
vm/transaction_execution/5 169.4±12.75ms 159.9±4.80ms -5.61%
vm/transaction_execution/50 145.4±8.52ms 140.3±6.86ms -3.51%

@simonjiao simonjiao merged commit 00a37d8 into dag-master Aug 7, 2024
5 checks passed
@simonjiao simonjiao deleted the fix-batchdbwriter branch August 7, 2024 14:11
@coderabbitai coderabbitai bot mentioned this pull request Nov 27, 2024
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants