Chore/migrate peering #473
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thank you for submitting this PR!
Getting other community members to do a review would be great help too on complex PRs (you can ask in the chats/forums). If you are unsure about something, just leave us a comment.
We currently aim to provide initial feedback/triaging within two business days. Please keep an eye on any labelling actions, as these will indicate priorities and status of your contribution. |
Codecov Report
@@ Coverage Diff @@
## main #473 +/- ##
==========================================
- Coverage 65.90% 65.79% -0.11%
==========================================
Files 205 203 -2
Lines 25185 25301 +116
==========================================
+ Hits 16598 16647 +49
- Misses 7125 7185 +60
- Partials 1462 1469 +7
|
gammazero
added a commit
to ipfs/kubo
that referenced
this pull request
Sep 29, 2023
The peering package is being used outside kubo, so should be relocated to boxo. See ipfs/boxo#473
gammazero
added a commit
to ipfs/kubo
that referenced
this pull request
Sep 29, 2023
The peering package is being used outside kubo, so should be relocated to boxo. See ipfs/boxo#473
gammazero
added a commit
to ipfs/kubo
that referenced
this pull request
Oct 2, 2023
The peering package is being used outside kubo, so should be relocated to boxo. See ipfs/boxo#473
gammazero
added a commit
to ipfs/kubo
that referenced
this pull request
Oct 5, 2023
The peering package is being used outside kubo, so should be relocated to boxo. See ipfs/boxo#473
gammazero
force-pushed
the
chore/migrate-peering
branch
from
fc4d3bc to
32b57d3
Compare
gammazero
added a commit
to ipfs/kubo
that referenced
this pull request
Oct 13, 2023
The peering package is being used outside kubo, so should be relocated to boxo. See ipfs/boxo#473
gammazero
force-pushed
the
chore/migrate-peering
branch
from
32b57d3 to
9ed01be
Compare
Jorropo
reviewed
Oct 16, 2023
gammazero
added a commit
to ipfs/kubo
that referenced
this pull request
Oct 18, 2023
The peering package is being used outside kubo, so should be relocated to boxo. See ipfs/boxo#473
gammazero
added a commit
to ipfs/kubo
that referenced
this pull request
Oct 20, 2023
The peering package is being used outside kubo, so should be relocated to boxo. See ipfs/boxo#473
MVP for #6097 This feature will repeatedly reconnect (with a randomized exponential backoff) to peers in a set of "peered" peers. In the future, this should be extended to: 1. Include a CLI for modifying this list at runtime. 2. Include additional options for peers we want to _protect_ but not connect to. 3. Allow configuring timeouts, backoff, etc. 4. Allow groups? Possibly through textile threads. 5. Allow for runtime-only peering rules. 6. Different reconnect policies. But this MVP should be a significant step forward. This commit was moved from ipfs/kubo@978091a
Co-authored-by: Will <will.scott@protocol.ai> This commit was moved from ipfs/kubo@8e52c7f
* better name for timer * cancel context from within stop This commit was moved from ipfs/kubo@0551c4d
This commit was moved from ipfs/kubo@fe2b289
This commit was moved from ipfs/kubo@87a293f
This commit was moved from ipfs/kubo@17b3b02
While preserving some randomness. And add a test. This commit was moved from ipfs/kubo@e10289a
This commit was moved from ipfs/kubo@fcdf77c
This test takes a full minute to run, and I was honestly thinking my run of "go test -v" had simply hung, as I saw no output and no apparent resource usage. The least we can do is print a few log messages before the potentially long waits, to hint that we're still making progress. Each of these "Eventually" and "Never" calls ends up blocking the test for a few seconds at a time. This commit was moved from ipfs/kubo@5e0c8bb
This commit fixes the issue that the test of peering.PeeringService must check the connection of h3 but did not. This line seemed to be unintended. This commit was moved from ipfs/kubo@b3a6de8
* feat: added swarm peering command supporting add, ls and rm Co-authored-by: Steven Allen <steven@stebalien.com> This commit was moved from ipfs/kubo@a651045
This commit was moved from ipfs/kubo@92854db
…xts (#8563) * plumb through go-datastore context changes * update go-libp2p to v0.16.0 * use LIBP2P_TCP_REUSEPORT instead of IPFS_REUSEPORT * use relay config * making deprecation notice match the go-ipfs-config key * docs(config): circuit relay v2 * docs(config): fix links and headers * feat(config): Internal.Libp2pForceReachability This switches to config that supports setting and reading Internal.Libp2pForceReachability OptionalString flag * use configuration option for static relays * chore: go-ipfs-config v0.18.0 https://github.com/ipfs/go-ipfs-config/releases/tag/v0.18.0 * feat: circuit v1 migration prompt when Swarm.EnableRelayHop is set (#8559) * exit when Swarm.EnableRelayHop is set * docs: Experimental.ShardingEnabled migration This ensures existing users of global sharding experiment get notified that the flag no longer works + that autosharding happens automatically. For people who NEED to keep the old behavior (eg. have no time to migrate today) there is a note about restoring it with `UnixFSShardingSizeThreshold`. * chore: add dag-jose code to the cid command output * add support for setting automatic unixfs sharding threshold from the config * test: have tests use low cutoff for sharding to mimic old behavior * test: change error message to match the current error * test: Add automatic sharding/unsharding tests (#8547) * test: refactored naming in the sharding sharness tests to make more sense * ci: set interop test executor to convenience image for Go1.16 + Node * ci: use interop master Co-authored-by: Marcin Rataj <lidel@lidel.org> Co-authored-by: Marten Seemann <martenseemann@gmail.com> Co-authored-by: Marcin Rataj <lidel@lidel.org> Co-authored-by: Gus Eggert <gus@gus.dev> Co-authored-by: Lucas Molas <schomatis@gmail.com> This commit was moved from ipfs/kubo@52c177c
* update go-libp2p to v0.18.0
* initialize the resource manager
* add resource manager stats/limit commands
* load limit file when building resource manager
* log absent limit file
* write rcmgr to file when IPFS_DEBUG_RCMGR is set
* fix: mark swarm limit|stats as experimental
* feat(cfg): opt-in Swarm.ResourceMgr
This ensures we can safely test the resource manager without impacting
default behavior.
- Resource manager is disabled by default
- Default for Swarm.ResourceMgr.Enabled is false for now
- Swarm.ResourceMgr.Limits allows user to tweak limits per specific
scope in a way that is persisted across restarts
- 'ipfs swarm limit system' outputs human-readable json
- 'ipfs swarm limit system new-limits.json' sets new runtime limits
(but does not change Swarm.ResourceMgr.Limits in the config)
Conventions to make libp2p devs life easier:
- 'IPFS_RCMGR=1 ipfs daemon' overrides the config and enables resource manager
- 'limit.json' overrides implicit defaults from libp2p (if present)
* docs(config): small tweaks
* fix: skip libp2p.ResourceManager if disabled
This ensures 'ipfs swarm limit|stats' work only when enabled.
* fix: use NullResourceManager when disabled
This reverts commit b19f7c9eca4cee4187f8cba3389dc2c930258512.
after clarification feedback from
ipfs/kubo#8680 (comment)
* style: rename IPFS_RCMGR to LIBP2P_RCMGR
preexisting libp2p toggles use LIBP2P_ prefix
* test: Swarm.ResourceMgr
* fix: location of opt-in limit.json and rcmgr.json.gz
Places these files inside of IPFS_PATH
* Update docs/config.md
* feat: expose rcmgr metrics when enabled (#8785)
* add metrics for the resource manager
* export protocol and service name in Prometheus metrics
* fix: expose rcmgr metrics only when enabled
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* refactor: rcmgr_metrics.go
* refactor: rcmgr_defaults.go
This file defines implicit limit defaults used when Swarm.ResourceMgr.Enabled
We keep vendored copy to ensure go-ipfs is not impacted when go-libp2p
decides to change defaults in any of the future releases.
* refactor: adjustedDefaultLimits
Cleans up the way we initialize defaults and adds a fix for case
when connection manager runs with high limits.
It also hides `Swarm.ResourceMgr.Limits` until we have a better
understanding what syntax makes sense.
* chore: cleanup after a review
* fix: restore go-ipld-prime v0.14.2
* fix: restore go-ds-flatfs v0.5.1
Co-authored-by: Lucas Molas <schomatis@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
This commit was moved from ipfs/kubo@514411b
PR #9030 This commit was moved from ipfs/kubo@04651e3
Fixes: #9225 This commit was moved from ipfs/kubo@196887c
This commit was moved from ipfs/kubo@f12b372
This commit was moved from ipfs/kubo@cc3c224
Jorropo
force-pushed
the
chore/migrate-peering
branch
from
dc47671 to
41637c6
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Migrating
peeringservice from kubo. The peering package is used outside of kubo, and should be relocated into boxo so that kubo is not used as an IPFS library.