[WIP] feat: Basic IPNS support #539
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This focuses on creating tests for the `iroh get` CLI command. The Iroh API is also improved along the way. This required more work in the underlying API to make it more testable. What's tested here is everything *except* the actual IPFS behavior -- the focus is on the behavior of the CLI and API and its interactions with the filesystem. ## trycmd tests In `iroh/tests/cmd` you can see quite a few `.trycmd` files. These describes the command-line interaction. Stuff behind `$ is a command, there's a special `? failed` indicator if the command is considered to have failed, and the output of the command is shown. New are the `.out` and `.in` directories with the same names as the `.trycmd` files. These describe the filesystem before the command runs (may be missing), and the filesystem after the command has run. This way we can describe the effects of the `iroh get` command - directories and files are supposed to be created. We can also test failure scenarios where we refuse to overwrite a directory that already exists. #269 tracks various test cases. ## `get_stream` The `get` high level CLI method has been removed from the mockable `Api` trait (read on to see where it went). Instead, a more low level but still useful API method `get_stream` has been added to the `Api` trait. This gets a stream of relative paths and `OutType`, describing the directories and files you can create. The big difference with what was there before is that it returns relative paths and doesn't calculate final destination paths -- that's up to the user of the API. ## No `async_trait` macro for `Api` trait The interactions between the `Api` trait, `mockall` macro and `async_trait` were getting so hairy I couldn't figure out how to express things anymore once I wanted to add `get_stream`. For my sanity and also to learn better how this really works underneath, I've rewritten the `Api` trait to describe itself explicitly in terms of `(Local)BoxFuture` and `(Local)_BoxStream`. It's more verbose but functionally equivalent and I could express what I wanted. ## `ApiExt` trait The `ApiExt` trait is a trait that implements the high level `get` command. It puts everything together: it handles various error conditions, accesses the stream and then writes the stream to the filesystem. It's basically `iroh get`. The `ApiExt` trait is solely intended to contain default trait methods, and is automatically available when the `Api` contract is fulfilled (if you `use` it). Factored out `save_get_stream` from `getadd.rs` to be solely concerned with turning a stream into files and directories on the files system. That makes it possible to test its behavior in isolation. ## test fixture The `get` test fixture now mocks `get_stream` and returns a fake stream made from a `Vec`. This defines the actual stuff that the CLI writes to disk. ## relative_path Now depend on the [`relative-path` crate](https://crates.io/crates/relative-path) because what the stream returns are clearly relative paths, and we want to force the user to do something with them before being able to actually write stuff.
* feat: range request support * cr * tests & fixes * fix etag range headers * rebase cleanup
…iroh` commands (n0-computer#348) * docs: add full help text for `lookup`, `connect`, `get`, `p2p`, and `iroh` commands * move long descriptions into constants Co-authored-by: b5 <sparkle_pony_2000@qri.io>
…ature fix(deps): We use the serde feature from Url
Newer clippy wants the identifiers in the format string when possible. Boring change but why not.
This makes sure to await the future from async_channel::Sender::send. When not awaiting this only makes a struct and noting is ever sent. Caught by clippy really.
…puter#656) Updates the requirements on [sysinfo](https://github.com/GuillaumeGomez/sysinfo) to permit the latest version. - [Release notes](https://github.com/GuillaumeGomez/sysinfo/releases) - [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md) - [Commits](https://github.com/GuillaumeGomez/sysinfo/commits) --- updated-dependencies: - dependency-name: sysinfo dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Floris Bruynooghe <flub@n0.computer>
…er#657) Updates the requirements on [rlimit](https://github.com/Nugine/rlimit) to permit the latest version. - [Release notes](https://github.com/Nugine/rlimit/releases) - [Changelog](https://github.com/Nugine/rlimit/blob/master/CHANGELOG.md) - [Commits](Nugine/rlimit@v0.8.3...v0.8.3) --- updated-dependencies: - dependency-name: rlimit dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Floris Bruynooghe <flub@n0.computer>
…uter#659) Updates the requirements on [base64](https://github.com/marshallpierce/rust-base64) to permit the latest version. - [Release notes](https://github.com/marshallpierce/rust-base64/releases) - [Changelog](https://github.com/marshallpierce/rust-base64/blob/master/RELEASE-NOTES.md) - [Commits](marshallpierce/rust-base64@v0.13.1...v0.13.1) --- updated-dependencies: - dependency-name: base64 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Floris Bruynooghe <flub@n0.computer>
The store does not use the RPC client at all. Remove it.
We used to use the same config for the service and server (aka the binary). This is confusing when creating configs to use with e.g. iroh-one, iroh-embed or iroh-share because some fields are not used. This splits off the config structs to avoid this problem, services now have a Config and binaries a ServerConfig. This allows creating the services standalone without all the baggage a server needs. While this isn't many fields yet, this will get worse as we add more features (this is split off from another PR where this seemed useful).
This removes the use of mockall which is causing a lot of trouble with the cargo features intricacies it brings with it. A lot of the tooling struggles with these different versions of the structs which also behave differently in surprising ways. The tests affected by this will be converted into end-to-end tests as part of is now unused. This is possibly only temprorary and that might come back when the end-to-end tests are added.
* feat: Add pub ctor to Directory * fix: reduce logging level * feat: Pretty printing for addrs * fix: clippy * pr fixes
ppodolsky
force-pushed
the
feature/ipns
branch
5 times, most recently
from
3473d39
to
3c5ff3f
Compare
ppodolsky
force-pushed
the
feature/ipns
branch
from
3c5ff3f
to
80dc47f
Compare
|
Hi, just a heads-up that if you are implementing things, and Wrote initial cleanup plan in ipfs/specs#376, there will be IPIP with details, but wanted to give you early ping on this. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've found planned refactorings in GetRecords libp2p/rust-libp2p#2712 (CC: @dignifiedquire) to make it streaming. In this PR I've supported DhtGet queries to iroh-p2p and gateway. Proposed changes take into account further possible changes in libp2p::GetRecords.
All implemented APIs return
Stream<PeerRecord>but streaming is not used yet, I'm just doing usual Quorum query, wait for response and send all received records through stream. It works, but there is also a lot of unwraps() and todo!() that I'll eliminate if general approach would be reviewed and accepted here.