Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Back off improvement #60

Merged
merged 104 commits into from
Jan 3, 2025
Merged

Back off improvement #60

merged 104 commits into from
Jan 3, 2025

Conversation

jost-s
Copy link
Contributor

@jost-s jost-s commented Dec 20, 2024 •
edited
Loading

Alternative to #56

closes #56

jost-s added 30 commits November 28, 2024 13:20
@jost-s
feat: implement basic fetch queue type
ea8db34
@jost-s
refactor: move trait to api & clean up
43c5f25
@jost-s
feat: add fetch task skeleton
69c7ba5
@jost-s
feat: add batching to fetch task
06d207e
@jost-s
docs: udpate fetch queue description
169fefb
@jost-s
style: fix lints
1042220
@jost-s
refactor: factorize fetch queue
d06a6c1
@jost-s
refactor: add config and builder
b85f33e
@jost-s
feat: add request loop logic & test
140537b
@jost-s
refactor: index by op id instead of agent id
db9de86
@jost-s
feat: cancel pause when ops are added
1f47a0f
@jost-s
Merge branch 'main' of https://github.com/holochain/kitsune2 into fea…
72e8298 
…t/fetch-pool
@jost-s
refactor: fetch ops individually with a simple queue
9d268bd
@jost-s
refactor: rename FetchQueue to Fetch
97ca843
@jost-s
Merge branch 'main' of https://github.com/holochain/kitsune2 into fea…
f8161cd 
…t/fetch-pool
@jost-s
refactor: rename package to fetch
ff42a22
@jost-s
docs: update fetch specs
d37fed6
@jost-s
test: add test for parallel request count
ffb0a9b
@jost-s
test: add multi op multi agent test
014348f
@jost-s
fix lints
765715d
@jost-s
refactor: move fetch into core crate
4c042f1
@jost-s
refactor(fetch): reinstate loop waking mechanism
5749858
@jost-s
fix(fetch): pause bf re-adding op to give other requests a chance to …
4ce895b 
…be processed
@jost-s
test: fix flaky tests
7518203
@jost-s
Merge branch 'main' of https://github.com/holochain/kitsune2 into fea…
a3a005f 
…t/fetch-pool
@jost-s
style: fix lints
129f3b9
@jost-s
style: fix docs
9109961
@jost-s
test: increase timeout for slow windows
865e7b4
@jost-s
test: increase timeout - that clearly is too long even for slow windows
9f57aaf
@jost-s
feat: add cool down list for unresponsive agents
1c0bd1a
jost-s added 14 commits December 16, 2024 15:40
@jost-s
refactor: simplify enc/dec test
c024fad
@jost-s
test: fix typos
e41dbb1
@jost-s
refactor: put unresponsive agents on exponential back off
b20c5c7
@jost-s
refactor: check for max back off at the end
3f0f7c8
@jost-s
refactor: set vec capacity for fetch tasks
abda1a9
@jost-s
refactor: simplify request retention
d7bc3b5
@jost-s
Merge branch 'main' of https://github.com/holochain/kitsune2 into ref…
bba7e78 
…actor/fetch-improvements
@jost-s
refactor: rename fn is_agent_backing_off to is_agent_on_back_off
d70a6da
@jost-s
fix: remove request when max back off interval expired
ff61632
@jost-s
refactor: change back_off_interval to 20 s
1236525
@jost-s
test: increase timeout due to flakiness
5499de9
@jost-s
wip
59880ef
@jost-s
Merge branch 'main' of https://github.com/holochain/kitsune2 into ref…
ac416d3 
…actor/improve-fetch-back-off-backon
@jost-s
fix: remove all requests from set when agent has been backed off max
c184187
@jost-s jost-s force-pushed the refactor/improve-fetch-back-off-backon branch from c53d082 to d985b95 Compare December 20, 2024 03:56
@jost-s jost-s force-pushed the refactor/improve-fetch-back-off-backon branch from d985b95 to 7cae7e2 Compare December 20, 2024 03:57
ThetaSinner
ThetaSinner previously approved these changes Dec 20, 2024
View reviewed changes
Copy link
Member

@ThetaSinner ThetaSinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I find that logic much easier to read even though the library is only hiding a little bit. I can forget about the maths and details and just focus on the logic

@jost-s jost-s marked this pull request as ready for review December 20, 2024 16:15
@jost-s
Copy link
Contributor Author

jost-s commented Dec 20, 2024

@ThetaSinner Moved back off to a separate module. The fetch module was getting too long.

@jost-s jost-s enabled auto-merge (squash) December 20, 2024 16:17
@jost-s jost-s mentioned this pull request Dec 23, 2024
Closed
@jost-s jost-s mentioned this pull request Jan 3, 2025
Open
@jost-s jost-s changed the title Alternative back off improvement Back off improvement Jan 3, 2025
@jost-s jost-s merged commit d9b7132 into main Jan 3, 2025
5 checks passed
@jost-s jost-s deleted the refactor/improve-fetch-back-off-backon branch January 3, 2025 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ThetaSinner ThetaSinner ThetaSinner approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jost-s @ThetaSinner