Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mechanism to remove idle workers #3370

Merged

Conversation

ljoss17
Copy link
Contributor

@ljoss17 ljoss17 commented May 25, 2023

Closes: #3385

Description


PR author checklist:

  • Added changelog entry, using unclog.
  • Added tests: integration (for Hermes) or unit/mock tests (for modules).
  • Linked to GitHub issue.
  • Updated code comments and documentation (e.g., docs/).
  • Tagged one reviewer who will be the one responsible for shepherding this PR.

Reviewer checklist:

  • Reviewed Files changed in the GitHub PR explorer.
  • Manually tested (in case integration/unit/mock tests are absent).

@ljoss17 ljoss17 marked this pull request as ready for review June 1, 2023 15:49
crates/relayer/src/supervisor.rs Outdated Show resolved Hide resolved
crates/relayer/src/worker/handle.rs Outdated Show resolved Hide resolved
crates/relayer/src/worker/map.rs Outdated Show resolved Hide resolved
crates/relayer/src/worker/map.rs Outdated Show resolved Hide resolved
crates/relayer/src/worker/packet.rs Outdated Show resolved Hide resolved
ljoss17 and others added 13 commits June 5, 2023 13:19
Co-authored-by: Romain Ruetschi <romain@informal.systems>
Signed-off-by: Luca Joss <43531661+ljoss17@users.noreply.github.com>
Co-authored-by: Romain Ruetschi <romain@informal.systems>
Signed-off-by: Luca Joss <43531661+ljoss17@users.noreply.github.com>
Co-authored-by: Romain Ruetschi <romain@informal.systems>
Signed-off-by: Luca Joss <43531661+ljoss17@users.noreply.github.com>
…ons (eg. Tendermint 0.35 and 0.36) (#3404)

* Use CompatMode::V0_34 when unsupported Tendermint version is detected

* Use CompatMode::V0_37 when unsupported Tendermint version is detected

* Fix warning when using fallback v0.37 CompatMode for Tendermint

* Fix typo

Signed-off-by: Romain Ruetschi <romain.ruetschi@gmail.com>

---------

Signed-off-by: Romain Ruetschi <romain.ruetschi@gmail.com>
Co-authored-by: Luca Joss <luca@informal.systems>
Co-authored-by: Romain Ruetschi <romain@informal.systems>
…-workers.md

Signed-off-by: Romain Ruetschi <romain.ruetschi@gmail.com>
@romac romac merged commit c5a5ce7 into luca_joss/disable-scan-when-no-startup-clear Jun 13, 2023
@romac romac deleted the luca_joss/clean-idle-workers branch June 13, 2023 15:09
romac added a commit that referenced this pull request Jun 14, 2023
…bled and `clear_on_start = false` (#3365)

* Only scan chains if 'clear_on_start=true'

* Add supervisor tests when scanning is enabled and disabled

* Add fees to ibc-transfer, required by Osmosis

* Improve supervisor tests with and without chain scan

* Add packet clearing test when scan is disabled

* Fix clearing without scan test

* Only scan when clear_on_start option is enabled, or client refresh or misbehavior workers are enabled, or the --full-scan argument is enabled

* Update no scan tests

* Disable channel and connection workers as well

* Add mechanism to remove idle workers (#3370)

* Add mechanism to remove idle workers

* Add integration test for worker cleanup mechanism

* Add idle channel worker integration test

* Fix CI job

* Improve worker cleanup test logs. Add changelog entry

* Update crates/relayer/src/worker/handle.rs

Co-authored-by: Romain Ruetschi <romain@informal.systems>
Signed-off-by: Luca Joss <43531661+ljoss17@users.noreply.github.com>

* Update crates/relayer/src/worker/map.rs

Co-authored-by: Romain Ruetschi <romain@informal.systems>
Signed-off-by: Luca Joss <43531661+ljoss17@users.noreply.github.com>

* Update crates/relayer/src/worker/map.rs

Co-authored-by: Romain Ruetschi <romain@informal.systems>
Signed-off-by: Luca Joss <43531661+ljoss17@users.noreply.github.com>

* Apply github suggestions and fix error

* Reduce idle worker cleanup frequency

* Increase time between idle worker cleanup checks

* Use CometBFT 0.37 compatibility mode for unsupported Tendermint versions (eg. Tendermint 0.35 and 0.36) (#3404)

* Use CompatMode::V0_34 when unsupported Tendermint version is detected

* Use CompatMode::V0_37 when unsupported Tendermint version is detected

* Fix warning when using fallback v0.37 CompatMode for Tendermint

* Fix typo

* Small cleanup

* Fix issue which spawned 2 supervisors in clean_workers tests

* Remove assert which can cause 'test_clean_channel_workers' to fail wrongly

* Update .changelog/unreleased/improvements/ibc-relayer/10-cleanup-idle-workers.md

* Add documentation on no scan startup

* Add changelog entry

---------

Co-authored-by: Romain Ruetschi <romain@informal.systems>
Co-authored-by: Sean Chen <seasn@informal.systems>
Co-authored-by: Luca Joss <luca@informal.systems>
@seanchen1991 seanchen1991 mentioned this pull request Jul 11, 2023
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants