Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add blpop cmd #373

Closed
wants to merge 9 commits into from
Closed

Conversation

saz97
Copy link

@saz97 saz97 commented Jul 9, 2024

Summary of Changes
This pull request implements the BLPOP command. The BLPOP command is designed to block the client when the specified key does not contain any data, and it will unblock the client once the key has data input.

Current Implementation
The basic framework for the BLPOP command has been established.
The command is functional in blocking the client and waiting for data input on the specified key.

Tasks Remaining

  • Raft log Handling: Implementation for handling binary logs needs to be completed.

  • Expiration Support: Add support for key expiration.

  • Locking Mechanism: Implement a locking mechanism to ensure thread safety and data consistency.

  • Multi-DB Consideration: Ensure the BLPOP command works correctly across multiple databases.

  • BRPOP Command: Implement the BRPOP command, which should function similarly to BLPOP but operates on the right end of the list.

Summary by CodeRabbit

  • 新功能

    • 添加了 ServeAndUnblockConns 方法,用于根据键值服务并解锁连接。
    • 引入了 BLPopCmd 类,支持从列表中弹出元素,并处理客户端阻塞。
    • PikiwiDB 类中新增 ScanExpiredBlockedConnsOfBlrpop 方法,用于扫描和处理过期的阻塞连接。
    • PikiwiDB 类增加了 GetMapFromKeyToConns 方法和 key_to_blocked_conns_ 成员变量,用于存储键到连接的映射。
  • 优化

    • PikiwiDB 类的 Init 函数中添加了一个定期任务,每250毫秒调用一次 ScanExpiredBlockedConnsOfBlrpop 方法。

Copy link

coderabbitai bot commented Jul 9, 2024

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

此次更新主要为BaseCmdPikiwiDB类引入了新的方法,用于处理和管理基于键的连接和过期连接。此外,新增了BLPopCmd类来处理从列表中弹出元素的命令。整个变更提升了系统对连接和命令处理的灵活性和效率。

Changes

文件 更改摘要
src/base_cmd.cc BaseCmd类中添加了ServeAndUnblockConns方法,用于根据键服务并解除连接。
src/cmd_list.cc 添加了BLPopCmd类,包含命令执行及客户阻塞方法,以及引入了pikiwidb.h头文件。
src/pikiwidb.cc PikiwiDB类中添加了ScanExpiredBlockedConnsOfBlrpop方法,用于处理过期连接。
src/pikiwidb.h PikiwiDB类中添加了GetMapFromKeyToConns方法和key_to_blocked_conns_成员变量。

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant BLPopCmd
    participant BaseCmd
    participant PikiwiDB
    
    Client->>+BLPopCmd: 执行BLPop命令
    BLPopCmd->>+BaseCmd: ServeAndUnblockConns(key)
    BaseCmd->>+PikiwiDB: 获取键到连接的映射
    PikiwiDB-->>-BaseCmd: 返回连接列表
    BaseCmd-->>-BLPopCmd: 处理连接并返回结果
    BLPopCmd-->>-Client: 返回元素或等待消息
    
    loop 每250毫秒
        PikiwiDB->>PikiwiDB: ScanExpiredBlockedConnsOfBlrpop
    end
Loading

Poem

🎉 在代码的森林里,我们迎来新变化,

🐰 连接解锁不再需要挣扎,

📜 命令执行也变得更轻松,

⏳ 过期连接定期清理,

🌟 PikiwiDB,如此强大,

让我们一同欢呼吧!


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot added the ✏️ Feature New feature or request label Jul 9, 2024
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 91e4863 and 2b7231f.

Files selected for processing (6)
  • src/base_cmd.cc (1 hunks)
  • src/base_cmd.h (2 hunks)
  • src/cmd_list.cc (3 hunks)
  • src/cmd_list.h (1 hunks)
  • src/cmd_table_manager.cc (1 hunks)
  • src/pikiwidb.h (2 hunks)
Additional comments not posted (12)
src/pikiwidb.h (2)

48-50: 添加的GetMapFromKeyToConns方法

这个方法返回了一个对key_to_blocked_conns_成员变量的引用。这个方法的实现是正确的,但需要注意线程安全问题。如果多个线程可能同时访问或修改这个映射,应该考虑添加适当的同步机制。

请确认在使用这个方法的地方是否已经考虑了线程安全问题。


67-74: 添加的key_to_blocked_conns_成员变量

这个成员变量用于存储从“Blockkey”到被阻塞的客户端连接的映射。注释详细说明了其用途,代码实现看起来没有问题。

这个成员变量的添加是合理的,与BLPop功能的描述一致。

src/cmd_list.h (2)

34-37: 新增的BLPopCmd类

这个类继承自BaseCmd,并添加了构造函数。构造函数接受命令名称和参数数量,符合命令类的设计模式。

这个构造函数的实现是正确的。


38-44: BLPopCmd类的方法实现

DoInitialDoCmd方法是命令执行的核心,BlockThisClientToWaitLRPush方法用于阻塞客户端等待数据。需要确保这些方法的逻辑正确,并且线程安全。

请确认这些方法的实现是否已经过测试,特别是BlockThisClientToWaitLRPush方法的阻塞逻辑。

src/base_cmd.cc (1)

99-125: 新增的ServeAndUnblockConns方法

这个方法遍历等待列表并为每个阻塞的客户端提供数据。需要注意几点:

  1. 确保在遍历等待列表时没有数据竞争。
  2. 处理所有可能的错误情况。
  3. 确保性能没有明显问题。

代码逻辑看起来是正确的,但需要确认在实际使用中是否表现良好。

请确认这个方法在高并发情况下的表现,并确保没有数据竞争问题。

src/cmd_table_manager.cc (1)

160-160: 在命令表中新增BLPop命令

这个命令的注册看起来是正确的,并且与其他命令的注册方式一致。

这个命令的注册实现是正确的。

src/cmd_list.cc (5)

32-32: 确认 ServeAndUnblockConns 方法的正确性

请确保 ServeAndUnblockConns 方法正确实现并集成。


120-121: BLPopCmd 类的构造函数

构造函数中初始化了命令名称和参数数量,代码看起来没有问题。


123-125: DoInitial 方法

该方法正确地设置了客户端的键。


128-140: DoCmd 方法的逻辑

请确认以下几点:

  1. LPop 操作是否正确处理了键不存在的情况。
  2. BlockThisClientToWaitLRPush 方法是否正确实现了阻塞逻辑。

142-152: BlockThisClientToWaitLRPush 方法

该方法将客户端添加到等待列表中,请确保以下几点:

  1. key_to_conns_ 的正确初始化。
  2. 并发情况下的线程安全性。
src/base_cmd.h (1)

319-320: ServeAndUnblockConns 方法

请确保 ServeAndUnblockConns 方法正确实现并集成到 BaseCmd 类中。

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 2b7231f and 0b0d9ac.

Files selected for processing (8)
  • src/base_cmd.cc (2 hunks)
  • src/base_cmd.h (3 hunks)
  • src/cmd_list.cc (3 hunks)
  • src/cmd_list.h (1 hunks)
  • src/cmd_table_manager.cc (1 hunks)
  • src/pikiwidb.cc (2 hunks)
  • src/pikiwidb.h (2 hunks)
  • tests/assets/default.conf (4 hunks)
Files skipped from review as they are similar to previous changes (5)
  • src/base_cmd.cc
  • src/base_cmd.h
  • src/cmd_list.cc
  • src/cmd_list.h
  • src/pikiwidb.h
Additional comments not posted (9)
src/cmd_table_manager.cc (1)

160-160: 添加 BLPop 命令

代码中正确地将 BLPop 命令添加到了命令表中。

src/pikiwidb.cc (2)

127-142: 实现了 ScanExpiredBlockedConnsOfBlrpop 方法

这个方法遍历了所有阻塞的连接并检查它们是否过期。如果过期,则向客户端发送空字符串并写回响应。逻辑正确,但需要注意性能和潜在的 race condition。

请确保此方法在高并发情况下的性能,并检查是否需要额外的锁机制来防止数据竞争。


188-189: 定期调度任务

正确地将 ScanExpiredBlockedConnsOfBlrpop 方法添加到了定期调度任务中,每250毫秒执行一次。

tests/assets/default.conf (6)

13-13: 绑定到特定 IP 地址

确保绑定到正确的IP地址,以防止潜在的安全问题。


38-38: 增加数据库数量

将数据库数量从3增加到16。确保系统资源足够支持增加的数据库数量。

请确认增加数据库数量后,系统是否有足够的资源来处理。


318-322: RocksDB 实例数量配置

将 RocksDB 实例数量配置为3。确保这个配置适合你的负载和性能需求。


324-326: 小型压缩阈值配置

配置了小型压缩阈值。确保这个配置适合你的数据增长和压缩策略。


329-344: RocksDB 配置参数

增加了多个 RocksDB 配置参数。这些配置参数有助于调整性能。确保这些配置符合你的性能和资源需求。


346-348: Raft 配置

配置了 Raft 端口偏移。确保 Raft 配置正确并且不会与其他服务冲突。

@saz97 saz97 changed the title feat: Add bpop cmd feat: Add blpop cmd Jul 12, 2024
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 0b0d9ac and 7c6b0eb.

Files selected for processing (4)
  • src/base_cmd.cc (2 hunks)
  • src/cmd_list.cc (3 hunks)
  • src/pikiwidb.cc (2 hunks)
  • src/pikiwidb.h (2 hunks)
Files skipped from review as they are similar to previous changes (4)
  • src/base_cmd.cc
  • src/cmd_list.cc
  • src/pikiwidb.cc
  • src/pikiwidb.h

@Tangruilin Tangruilin marked this pull request as draft July 13, 2024 11:36
@AlexStocks AlexStocks closed this Aug 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✏️ Feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants