Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] feat: iroh-sync #1216

Closed
wants to merge 46 commits into from
Closed

[WIP] feat: iroh-sync #1216

wants to merge 46 commits into from

Commits on Aug 7, 2023

  1. feat: initial prototype of integrating iroh-sync

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    ac96d9d View commit details
    Browse the repository at this point in the history

  2. sync: start impl of multikey

    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    5039bd9 View commit details
    Browse the repository at this point in the history

  3. feat: integrate iroh-sync and iroh-gossip, add example

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    f1f5947 View commit details
    Browse the repository at this point in the history

  4. feat: make the live sync handler work with many docs

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    90d696e View commit details
    Browse the repository at this point in the history

  5. chore: cleanup and clippy

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    476896d View commit details
    Browse the repository at this point in the history

  6. chore: remove old code and add docs

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    1e505bc View commit details
    Browse the repository at this point in the history

  7. feat: WIP integration of sync and bytes 

    * removes content support from iroh-sync
* adds a quick-and-dirty writable database to iroh-bytes (will be
  replaced with a better generic writable database soon)
* adds a `Downloader` to queue get requests for individual hashes from
  individual peers
* adds a `BlobStore` that combines the writable db with the downloader
* adds a `Doc` abstraction that combines an iroh-sync `Replica` with a
  `BlobStore` to download content from peers on-demand
* updates the sync repl example to plug it all together
* also adds very basic persistence to `Replica` (encode to byte string)
  and uses this in the repl example
    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    2af3471 View commit details
    Browse the repository at this point in the history

  8. feat: proper REPL for sync example, and docs store 

    * make the REPL in the sync example work properly with rustyline for
  editing and reading input, shell-style argument parsing and clap for
  parsing commands
* add a docs store for opening and closing docs
* add author to doc struct
    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    3a671d2 View commit details
    Browse the repository at this point in the history

  9. feat(example-sync): add ticket command

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    97ae828 View commit details
    Browse the repository at this point in the history

  10. fix: adapt to changes on main after rebase

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    0e96ba9 View commit details
    Browse the repository at this point in the history

  11. example(sync): add watch command

    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    400be8f View commit details
    Browse the repository at this point in the history

  12. fixup

    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    df690d2 View commit details
    Browse the repository at this point in the history

  13. chore: unused variables

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    ce72dcc View commit details
    Browse the repository at this point in the history

  14. fix tests and warnings

    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    1ed0cca View commit details
    Browse the repository at this point in the history

  15. clippy cleanups

    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    7d24fda View commit details
    Browse the repository at this point in the history

  16. feat(iroh): metrics for iroh-sync

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    613260c View commit details
    Browse the repository at this point in the history

  17. fix: remove usage of unbounded channels 

    uses flume channels to allow for combined sync and async usage
    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    684b753 View commit details
    Browse the repository at this point in the history

  18. add todo

    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    9b9042e View commit details
    Browse the repository at this point in the history

  19. sync: implement more extensive fetch methods

    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    b2669e0 View commit details
    Browse the repository at this point in the history

  20. add prefix methods

    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    efb5405 View commit details
    Browse the repository at this point in the history

  21. feat: enable metrics server on sync (#1308)

    @Arqu @Frando
    Arqu authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    83ecf56 View commit details
    Browse the repository at this point in the history

  22. chore: update deny.toml

    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    6a4f83b View commit details
    Browse the repository at this point in the history

  23. fix clippy and feature selection

    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    9d95057 View commit details
    Browse the repository at this point in the history

  24. doc: more docs fixes

    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    de430b4 View commit details
    Browse the repository at this point in the history

  25. feat: file system export/import in sync repl

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    4f6ab65 View commit details
    Browse the repository at this point in the history

  26. feat: hammer sync example

    @Arqu @Frando
    Arqu authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    f0dc252 View commit details
    Browse the repository at this point in the history

  27. chore: fmt

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    4c4c31c View commit details
    Browse the repository at this point in the history

  28. allow specifying number of threads and print out some basic stats

    @Arqu @Frando
    Arqu authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    2177042 View commit details
    Browse the repository at this point in the history

  29. mad clippy is mad

    @Arqu @Frando
    Arqu authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    65bafe5 View commit details
    Browse the repository at this point in the history

  30. extend hammer with get/set modes

    @Arqu @Frando
    Arqu authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    d964d80 View commit details
    Browse the repository at this point in the history

  31. refactor: use clap::ValueEnum

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    53bd1c0 View commit details
    Browse the repository at this point in the history

  32. refactor: check results, and clearer variable names

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    dcf4c55 View commit details
    Browse the repository at this point in the history

  33. refactor: count actual rows, and make mode an argument

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    38cce3f View commit details
    Browse the repository at this point in the history

  34. fix: rebase fix

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    871aa06 View commit details
    Browse the repository at this point in the history

  35. feat(iroh-sync): implement file system backed for documents (#1315) 

    * start refactoring store into its own module

* implement more details

* works again

* draft fs db and integrate error handling

* fill out more of the implemenation

* lifetime sadness

* self referential fight: Rust 0 - Dig 1

* basic tests and range fixes

* introduce Store trait and update tests to test against both impls

* implement remove

* integrate new storage into the example

* implement iterators

* fixes and more tests

* clippy and deny cleanup
    @dignifiedquire @Frando
    dignifiedquire authored and Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    6a42b69 View commit details
    Browse the repository at this point in the history

  36. adapt after rebase on main

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    7c6cb35 View commit details
    Browse the repository at this point in the history

  37. chore: fmt

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    263fa14 View commit details
    Browse the repository at this point in the history

  38. feat: download from peer that informed us about a change

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    dd8cef8 View commit details
    Browse the repository at this point in the history

  39. refactor: move downloader out of sync module

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    ed8cb2d View commit details
    Browse the repository at this point in the history

  40. fix: rebase

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    e4e6bc8 View commit details
    Browse the repository at this point in the history

  41. fix imports

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    bafdc63 View commit details
    Browse the repository at this point in the history

  42. chore: fmt

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    1cf78b2 View commit details
    Browse the repository at this point in the history

  43. fix: metrics

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    a22c9e7 View commit details
    Browse the repository at this point in the history

  44. Merge pull request #1314 from n0-computer/sync-download-from-prev-peer 

    [WIP] feat(iroh-sync): download from peer that informed us about a change
    @Frando
    Frando authored Aug 7, 2023
    Configuration menu
    Copy the full SHA
    b37aac4 View commit details
    Browse the repository at this point in the history

  45. fix: feature flags

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    3fc96bd View commit details
    Browse the repository at this point in the history

  46. chore: fmt

    @Frando
    Frando committed Aug 7, 2023
    Configuration menu
    Copy the full SHA
    9f12429 View commit details
    Browse the repository at this point in the history