…nt (n0-computer#317)

use store exported path in iroh-one

closes #255, closes #309

* feat(gateway): fetch a DAG recursively as CAR.

* test(gateway): add test for recursive car fetch

also fixes clippy and code style issues

* refactor(resolver): use mapper in recursive resolve

* style(resolver): fix comment

* style(resolver): clippy fix

* style(gateway): clippy fix and test fix

* fix(unixfs): ensure links get persisted in store

This had to be merged with some other changes related to the size info.
Introduce a Block struct that has data, hash and links for DRY.

* test(unixfs): validate blocks in all roundtrip tests

Validate blocks in all roundtrip tests to make sure that the blocks that
are returned from file.encode or dir.encode are internally consistent.

* fix(store): only create new id when needed

The old code was creating new ids even if we already knew the cid, which
caused some havoc.

This allows adding large files without getting an error. It is a stopgap until
we have a persistent kad store.

## move RPC client construction into `iroh` (read: `iroh-api`)

I've moved creation of the RpcClient away from the CLI into `iroh` (to be `iroh-api`). This means that the API gains a bunch of dependencies and the CLI loses them.

Doing this made me realize that the api should own the rpc client, so changed from a reference (with a lifetime) to ownership.

## `iroh get` start of implementation

Start of implementation of `iroh get` with a few mocked CLI tests

## `iroh` -> `iroh-api` crate

The plan is to have an `iroh` crate that contains the CLI and an `iroh-api` crate that contains the Rust API. Here we've done half of that renaming: `iroh` is renamed to `iroh-api`. I haven't done the second half, renaming `iroh-ctl` to `iroh`, as that would make this commit too different to read.

* Rename the iroh-ctl crate to the iroh crate

* test: test --version flag does what it should now

Acknowledges that a `raw` node is a valid `UnixfsNode` and therefore can be resolved as a file in the gateway

This reverts commit 6b34a80.

… project (n0-computer#334)

This tripped up some test fixtures I wrote where I had inadvertently created
a directory `foo` and hadn't realized they were never actually checked in. Was a bit
of a headscratcher we want to prevent in the future.

also reverts previous commit that made a bad assumption about allowing `Raw` as a `UnixfsType` in the resolver

…uter#333)

add support for manual pages generated from clap documentation, use the xtasks pattern to get "npm run" or "make"-like functionality

This makes the code less verbose, and also will be helpful if we can figure
out how to permanently store the cf handles.

* fix: increase concurrency limit

* feat: handle only if cached calls

* fix: handle head requests

* align accept headers with spec

* use resolver::Path instead of string paths

* clean up response format parsing

* cleanup etag check

* remove unused flags & config options

* better content type detection - part 1

* feat: better content type detection

* make clippy happy

* per spec content disposition & cache-control

* cleanup

* rebase cleanup

* cr

* cr

* rebase cleanup

This focuses on creating tests for the `iroh get` CLI command. The Iroh API is also improved along the way.

This required more work in the underlying API to make it more testable. What's tested here is everything *except* the actual IPFS behavior -- the focus is on the behavior of the CLI and API and its interactions with the filesystem.

 ## trycmd tests

In `iroh/tests/cmd` you can see quite a few `.trycmd` files. These describes the command-line interaction. Stuff behind `$ is a command, there's a special `? failed` indicator if the command is considered to have failed, and the output of the command is shown.

New are the `.out` and `.in` directories with the same names as the `.trycmd` files. These describe the filesystem before the command runs (may be missing), and the filesystem after the command has run. This way we can describe the effects of the `iroh get` command - directories and files are supposed to be created. We can also test failure scenarios where we refuse to overwrite a directory that already exists. #269 tracks various test cases.

## `get_stream`

The `get` high level CLI method has been removed from the mockable `Api` trait (read on to see where it went). Instead, a more low level but still useful API method `get_stream` has been added to the `Api` trait. This gets a stream of relative paths and `OutType`, describing the directories and files you can create. 

The big difference with what was there before is that it returns relative paths and doesn't calculate final destination paths -- that's up to the user of the API.

## No `async_trait` macro for `Api` trait

The interactions between the `Api` trait, `mockall` macro and `async_trait` were getting so hairy I couldn't figure out how to express things anymore once I wanted to add `get_stream`. For my sanity and also to learn better how this really works underneath, I've rewritten the `Api` trait to describe itself explicitly in terms of `(Local)BoxFuture` and `(Local)_BoxStream`. It's more verbose but functionally equivalent and I could express what I wanted. 

## `ApiExt` trait

The `ApiExt` trait is a trait that implements the high level `get` command. It puts everything together: it handles various error conditions, accesses the stream and then writes the stream to the filesystem. It's basically `iroh get`.

The `ApiExt` trait is solely intended to contain default trait methods, and is automatically available when the `Api` contract is fulfilled (if you `use` it).

Factored out `save_get_stream` from `getadd.rs` to be solely concerned with turning a stream into files and directories on the files system. That makes it possible to test its behavior in isolation.

## test fixture

The `get` test fixture now mocks `get_stream` and returns a fake stream made from a `Vec`. This defines the actual stuff that the CLI writes to disk.

## relative_path

Now depend on the [`relative-path` crate](https://crates.io/crates/relative-path) because what the stream returns are clearly relative paths, and we want to force the user to do something with them before being able to actually write stuff.

* feat: range request support

* cr

* tests & fixes

* fix etag range headers

* rebase cleanup

…omputer#338)

…iroh` commands (n0-computer#348)

* docs: add full help text for `lookup`, `connect`, `get`, `p2p`, and
`iroh` commands

* move long descriptions into constants

Co-authored-by: b5 <sparkle_pony_2000@qri.io>

This contains work on the add command, with a few CLI tests and some API tweaks.

## add layering

`add_file` and `add_dir` in the core `Api` that's mockable, and the slightly higher level `add` is above it in `ApiExt`.

## recursive simplication

Test for the `-r` flag in the CLI and just let `add_dir` in the resolver add a directory without worrying. This is tested.

## a few add CLI tests

A few basic add CLI tests.

## CLI test cleanup

I've went through the `cli_tests.rs` module step by step cleaning and reorganizing the tests so it's more easy to maintain them. Unfortunately `trycmd` gets too finicky if I try to use its globbing feature (as input and output directories seem shared sometimes). So now I maintain an explicit list in the same order as they show up alphabetically. It's too verbose but it's the best we can do for now.