Grandcentral Routing (temporary while DHT is shown to be correct)

[btc]

@jbenet @mappum @whyrusleeping

TODOs

• pick a serialization format for provider records. perhaps just proto message
• extract database Backend from grandcentral.Server create datastore
• pass config value (server | client) in from UX
• pick a peer/address/host for remote routing server
• finally, make this test pass: https://build.protocol-dev.com/job/ipfs-network-test-add-cat/

[whyrusleeping]

Whats this?

[whyrusleeping]

Ah! So this is a dht alternative using a central server to pull information from?

[jbenet]

@whyrusleeping this is a temporary routing system that we're putting in place so we can demo things correctly without depending on the dht (which does not work reliably,robustly, and permanently yet). We were demoing things last night and things weren't working well (e.g. not resolving properly), discussed this late last night (sorry, live, not on irc), and decided to use this so we can separate the concerns.

[whyrusleeping]

oh alright, I was really confused, lol

[jbenet]

@whyrusleeping sorry-- should've posted something here / on irc. (i guess this is that message? >.< )

[jbenet]

let's not call this tracker. tracker has specific connotations in p2p (namely BT trackers), which is related but significantly different. Let's call it "delegated" routing (you route through whatever the delegate says)

[jbenet]

@briantigerchow is this still ongoing? dht seems to be better off now, but i'd like to see some large (thousands of nodes) tests before really relying on it.

[btc]

@jbenet This is ready for some initial testing

[btc]

@jbenet removed the stray .go-ipfs dir and rebased/erased it out of the git history. declared the s3 datastore as thread safe. used the bootstrap strings from the config package.

[jbenet]

@briantigerchow changes lgtm, thank you.

[jbenet]

Everything LGTM! let me know how you want to proceed with this merge. My constraints are:

• trivial to manage
• easy for users to know how to switch between dht/snr (this could be just an example or a note in the readme we can point them to)
• we should not enable it by default until we're certain it is better than dht. (i.e. run it alongside for a while and we can play with it?). We could become certain of this after a few hours of trying it out

[btc]

Feel free to merge. The servers are already running. Users Pass flag "--routing=supernode" to turn it on.

[whyrusleeping]

We're getting that fault panic on tip a lot.

[btc]

Fault panic on tip?

[whyrusleeping]

@briantigerchow Yeah, on the jenkins "golang tip" build, we keep seeing this really weird fault, that we arent sure the cause of