Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

raft: provide protection against unbounded Raft log growth #131

Closed
siddontang opened this issue Oct 10, 2018 · 4 comments · Fixed by #398
Closed

raft: provide protection against unbounded Raft log growth #131

siddontang opened this issue Oct 10, 2018 · 4 comments · Fixed by #398
Labels
Enhancement An improvement to existing code. Good First Issue A good issue for a new contributor.

Comments

@siddontang
Copy link
Contributor

siddontang commented Oct 10, 2018 •
edited by Hoverbear
Loading

Introduce a new, optional max_uncommitted_entries configuration. This config limits the max number of uncommitted entries that may be appended to a leader's log. Once this limit is exceeded, proposals will begin to return ProposalDropped errors.

You can refer to refer etcd-io/etcd#10167

This issue can be closed by:

  • Modifying the Config to have this new configuration setting.
  • Modify Raft::handle_append_entries to check the limit. You can return the existing ProposalDropped error.
  • Adding a test to tests/integration_cases/test_raft.rs similar to the one in the etcd PR.

As always, if you'd like to tackle this, or need any help, please feel free to reach out! We'd love to get you involved.
@Hoverbear Hoverbear added Enhancement An improvement to existing code. Good First Issue A good issue for a new contributor. labels Oct 31, 2018
@Hoverbear
Copy link
Contributor

@utsl42 mentioned they might want to take a crack at this.

@utsl42 utsl42 mentioned this issue Nov 8, 2018
Closed
@Fullstop000 Fullstop000 mentioned this issue Mar 17, 2019
Closed
@Ishiihara
Copy link
Contributor

@siddontang @Hoverbear I am interested in taking this issue, can you assign this to me?

@Hoverbear
Copy link
Contributor

@Ishiihara Please consider yourself assigned! (We can't assign you an issue on github until you have a merge I think...)

Do you think you could start from #139, or are you planning to start from scratch?

c0x0o pushed a commit to c0x0o/raft-rs that referenced this issue Sep 20, 2020
@c0x0o
*: add protection against unlimited log growth(tikv#131)
680b93c 
- Add protection against uncommitted log growth of leader.
  This protection is done by estimate size of entry data
  rather than count of entry. So entry with no data will
  never be dropped
- Add tests for raft and raw_node

Signed-off-by: lyzongyuan <lyzongyuan@gmail.com>
@c0x0o
Copy link
Contributor

c0x0o commented Sep 20, 2020 •
edited
Loading

Hi, i made a new PR working on this issue and code is ready for review. Glad to join the community :) @Hoverbear @siddontang

@c0x0o c0x0o mentioned this issue Oct 16, 2020
Merged
BusyJay pushed a commit that referenced this issue Oct 26, 2020
@c0x0o
*: add protection against unlimited log growth(#131) (#398)
a424fdb 
- Add protection against uncommitted log growth of leader.
  This protection is done by estimate size of entry data
  rather than count of entry. So entry with no data will
  never be dropped
- Add tests for raft and raw_node

Signed-off-by: lyzongyuan <lyzongyuan@gmail.com>
jayzhan211 pushed a commit to jayzhan211/raft-rs that referenced this issue Oct 26, 2020
@c0x0o @jayzhan211
*: add protection against unlimited log growth(tikv#131) (tikv#398)
9c95e03 
- Add protection against uncommitted log growth of leader.
  This protection is done by estimate size of entry data
  rather than count of entry. So entry with no data will
  never be dropped
- Add tests for raft and raw_node

Signed-off-by: lyzongyuan <lyzongyuan@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Enhancement An improvement to existing code. Good First Issue A good issue for a new contributor.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

*: add protection against unlimited log growth(#131) c0x0o/raft-rs
4 participants
@Hoverbear @siddontang @Ishiihara @c0x0o