Simulating Bitswap

[kbala444]

@jbenet mentioned it would be good to create a simulator for bitswap to get data and test classification methods.

How should I go about starting something like this?

[jbenet]

We need something like https://github.com/whyrusleeping/dhtHell and https://github.com/whyrusleeping/iptb

ideally we would have a simulator that (in order of importance):

• can run scripts for "test scenarios", where we can simulate various workloads
  • star topology, one edge node adds a file, everyone else starts fetching the whole thing.
  • large file is added, hundreds of short-lived peers emerge to get it (flash mob)
  • random topology, one video source, n/N start downloading the whole file. some seek randomly.
  • and so on.
• could run bitswap in isolation (to ensure it works independent of the rest of ipfs)
• could run bitswap with full ipfs (iptb, to ensure it works with the rest of ipfs)
• can run many instances in the same process
• can run many instances across multiple processes
• can run instances across multiple machines (this would be really nice, but likely not needed)

I think the ideal plan of attack here is

• extract bitswap into own package
• mock network instrumentation (latency, bandwidth, etc) (@whyrusleeping)
• get raw bitswap working with a mock network
• produce a binary to test with (maybe one that runs just bitswap and uses stdin/stdout to communicate with user?)
• make script-running thing to test scenarios
• make a bunch of representative scenarios

it's possible we could cut corners, and either use full iptb for now, or something like dhtHell. my worry there is that we must test bitswap in isolation (in addition to i conjunction with the rest of ipfs). Without isolation we cannot test the model honestly.

after the above, we should be at a point where we can test + measure bitswap perf. once there, we can experiment with changing the decision engine, and other features. (this will be really valuable to test other things about bitswap).

cc @whyrusleeping

[whyrusleeping]

• extract bitswap into own package

waiting on me (or others) to write a nice packaging tool. I have it sketched and the basics of how i want it to work down.

• mock network instrumentation (latency, bandwidth, etc) (@whyrusleeping)

This code exists somewhere... i forget why i didnt merge it.

[whyrusleeping]

Alright, for a basic simulator, it would be cool to have a program similar to dhthell so we can script different scenarios, being able to set different factors such as number of nodes, initial connections, routing latency, message latency, bandwidth, etc will be good. The main commands youll have will just be 'want' and 'cancel' blocks. and maybe things like 'expect' or 'wait' to test for behaviour.

Take a look at the sample scripts in dhthell for some 'inspiration' and let me know if you have any questions.

[whyrusleeping]

heres a network simulator tool i was working on a while back: https://github.com/ipfs/go-ipfs/tree/feat/netsim

[kbala444]

Hey as I'm doing this I have a few questions.

By 'want' and 'cancel' do you mean actually interact with the WantManagers of test nodes? Like call WantBlocks and CancelWants? I currently have the simulation working similar to bitswap_test.go, utilizing the TestSessionGenerator and interacting with the Exchanges of the Instances (calling Exchange.HasBlock/WantBlock). Is this ok or is it too abstracted from what you wanted?

[jbenet]

Interacting with the Exhange API is good. We may want a bit closer access to some of the bitswap methods for scripting. (Maybe bitswap doesn't expose these at the moment) but that can probably come later as needed.

'Want' and 'Cancel' are part of the protocol definition in go-ipfs (not the paper yet sadly), they're sent as messages on the wire. Cancel is just a way of sending the other node diffs on the want list instead of sending the whole thing (which can be big) every time.

[jbenet]

link to sim: https://github.com/heems/bssim

[whyrusleeping]

thats looking good 👍 make sure to have configurable block sizes for chunking so we can easily simulate 'large' file trees.

[kbala444]

cool, added configurable block sizes