jsipfs add has defaults differing from ipfs add

[ribasushi]

In go-ipfs raw-leaves defaults to true when cid-version=1 is specified.
Additional the defaults for rabin are identical to rabin-262144 which translates to rabin-87381-262144-393216

However js-ipfs does otherwise, observe:

go	chunker:rabin	rl:	bafybeifgfx3nkx6lfyrzivtqmeqxuitds7t7k6t2anudptymhsfbmpsvzu
js	chunker:rabin	rl:	bafybeictpvoociikd5pm4vnqtbedccef5gohdjya6ly726csvwm6jy3xea
go	chunker:rabin	rl:--raw-leaves=true	bafybeifgfx3nkx6lfyrzivtqmeqxuitds7t7k6t2anudptymhsfbmpsvzu
js	chunker:rabin	rl:--raw-leaves=true	bafybeicos3bc7li5wahssh2qs2gsdokcjmce3wymusdm6jrmm6wvw6vf2i
go	chunker:rabin	rl:--raw-leaves=false	bafybeierguac6kvgj2plzgsqkpwkg2czmwwhcoccrkdsjgc5loucwjawje
js	chunker:rabin	rl:--raw-leaves=false	bafybeictpvoociikd5pm4vnqtbedccef5gohdjya6ly726csvwm6jy3xea
go	chunker:rabin-262144	rl:	bafybeifgfx3nkx6lfyrzivtqmeqxuitds7t7k6t2anudptymhsfbmpsvzu
js	chunker:rabin-262144	rl:	bafybeictpvoociikd5pm4vnqtbedccef5gohdjya6ly726csvwm6jy3xea
go	chunker:rabin-262144	rl:--raw-leaves=true	bafybeifgfx3nkx6lfyrzivtqmeqxuitds7t7k6t2anudptymhsfbmpsvzu
js	chunker:rabin-262144	rl:--raw-leaves=true	bafybeicos3bc7li5wahssh2qs2gsdokcjmce3wymusdm6jrmm6wvw6vf2i
go	chunker:rabin-262144	rl:--raw-leaves=false	bafybeierguac6kvgj2plzgsqkpwkg2czmwwhcoccrkdsjgc5loucwjawje
js	chunker:rabin-262144	rl:--raw-leaves=false	bafybeictpvoociikd5pm4vnqtbedccef5gohdjya6ly726csvwm6jy3xea
go	chunker:rabin-87381-262144-393216	rl:	bafybeifgfx3nkx6lfyrzivtqmeqxuitds7t7k6t2anudptymhsfbmpsvzu
js	chunker:rabin-87381-262144-393216	rl:	bafybeierguac6kvgj2plzgsqkpwkg2czmwwhcoccrkdsjgc5loucwjawje
go	chunker:rabin-87381-262144-393216	rl:--raw-leaves=true	bafybeifgfx3nkx6lfyrzivtqmeqxuitds7t7k6t2anudptymhsfbmpsvzu
js	chunker:rabin-87381-262144-393216	rl:--raw-leaves=true	bafybeifgfx3nkx6lfyrzivtqmeqxuitds7t7k6t2anudptymhsfbmpsvzu
go	chunker:rabin-87381-262144-393216	rl:--raw-leaves=false	bafybeierguac6kvgj2plzgsqkpwkg2czmwwhcoccrkdsjgc5loucwjawje
js	chunker:rabin-87381-262144-393216	rl:--raw-leaves=false	bafybeierguac6kvgj2plzgsqkpwkg2czmwwhcoccrkdsjgc5loucwjawje

Produced via:

#!/bin/bash

for ch in \
  "rabin" \
  "rabin-262144" \
  "rabin-87381-262144-393216" \
  ; do \
    for rl in \
      "" \
      "--raw-leaves=true" \
      "--raw-leaves=false"; do \
        echo -en "go\tchunker:$ch\trl:$rl\t"; dd if=/dev/zero bs=$((1024*1024)) count=16 2>/dev/null | cmd/ipfs/ipfs add -nq --cid-version=1 --chunker=$ch $rl
        echo -en "js\tchunker:$ch\trl:$rl\t"; dd if=/dev/zero bs=$((1024*1024)) count=16 2>/dev/null | jsipfs add -nq --cid-version=1 --chunker=$ch $rl
    done
done

[ribasushi]

Additional info requested by @hugomrdias to potentially incorporate a slimmed-down version of the rabin chunker into the wasm version

@hugomrdias aside from that: yes: the rabin chunker as written here is complete, and there isn't much more left to squeeze out
note that my "intake" interface is slightly different due to being a radically changed framework, so it will take a little massaging to retrofit
instead of readers, it expects a buffer and an indication whether "more is coming" or not
chunkCB is where it passes each chunk as it finds them
https://github.com/ipfs/DAGger/blob/19fc1fd0e8/dagger/chunker/ipfs/rabin/rabin.go#L28-L89

@hugomrdias note I have not looked at the wasm version, so I can't guarantee this will be better, but it likely can be ( I do away with the window buffer entirely )

[achingbrain]

Something that may cause a problem here is that the default polynomial value of 17437180132763653 used by go-IPFS with the rabin chunker is a bigger number than you can represent in JS:

It needs to use long or bignumber.js BigInt to represent it instead.