Usage - How to setup/init a repository?

[pruflyos]

Hi,
the usage descriptions are very limited. git clone ipld::20dae521ef399bcf95d4ddb3cefc0eeb49658d2a somewhat works (it doesn't finish and hangs at some point).

So how do I initialize my own repository to push it via IPLD?
git init and then git push ipld:: doesn't do the trick! I need to configure the remote master somehow. How do I calculate the hash for my repository?

Thanks!

[victorb]

You should be able to do git push ipld:: master and it should show you the hash. I've tried this, but unfortunately seems the not work anymore, with latest go-ipfs at least.

go-ipfs version: 0.4.14-dev-92226a1ea
Repo version: 6
System version: amd64/linux
Golang version: go1.9.3

$ git push ipld:: master
< capabilities
> push
> fetch
>
< list for-push
> 0000000000000000000000000000000000000000 refs/heads/master
> @refs/heads/master HEAD
>
< push refs/heads/master:refs/heads/master
<
Processsing tasks
2018/02/01 10:58:04 command push: push: dag/put: no parser for format "git" using input type "raw"
error: failed to push some refs to 'ipld::'

(btw, got it from the demo from a all-hands that starts at 13:50 in this recording: https://ipfs.io/ipfs/QmQtTekDaG4475fqv8oVKanLo11C8rSKeVcbKee28WR1zd/2017-08-07%2012.03.58%20IPFS%20All%20Hands%20Call%20779351365.mp4 )

Also, the fetch of the clone command in the readme seems to never complete, left it running for hours and it's still fetching things...

[pruflyos]

Thanks for the video. I got it working now! :-) I forgot to do a git commit -m "Init" after git init and before trying to push.

mkdir test
cd test
git init
touch test.txt
git add test.txt
git commit -m "initial test"
git push ipld:: master

Also, the fetch of the clone command in the readme seems to never complete, left it running for hours and it's still fetching things...

For me it just stops fetching after a long time... but it doesn't finish. If you abort with Ctrl+c the folder is gone again and it starts from scratch the next time until it stops (I guess at the same file). Maybe the demo source is not fully available on IPFS anymore and instead of a timeout, it keeps waiting forever until the rest of the data is available on at least one node again?

[magik6k]

Also, the fetch of the clone command in the readme seems to never complete, left it running for hours and it's still fetching things...

Do you have/had files larger than 2mb in the repo? If yes then this is a limitation of ipfs (maximum block size is set to 2mb, larger blocks will not be transferred with bitswap).

One idea I have to fix that is to put some sort of dictionary in root of the ipns repos (they are WIP, this feature wold be really useful)

[victorb]

@magik6k I'm using the exact command you have in the readme, I don't know how that repository is structured at all.

git clone ipld::20dae521ef399bcf95d4ddb3cefc0eeb49658d2a from https://github.com/magik6k/git-remote-ipld/blob/master/README.md#usage

[pruflyos]

Anyway, I'm super stoked! IPFS + Git is one of the killer use cases for IPFS right now!! Keep up the great work!

[pruflyos]

Is it possible to get/find out the IPFS Hash (starting with "Qm...") of a record in IPLD?
Use case: I commit a README.md file via git ipld and I want to point people to a gateway like https://ipfs.io/ipfs/Qm... so they can read/view the latest version directly, without having to install ipfs-go and git-remote-ipld?

[magik6k]

So the example hash (which pointed at go-ipfs repo) had 3 objects larger than the limit.. I updated it in f2fc4db (now it points at ipfs/ipfs which should work fine)

[magik6k]

Is it possible to get/find out the IPFS Hash (starting with "Qm...") of a record in IPLD

There is no direct mapping between the two - you need a separate viewer. One WIP thing is called IGiS - in-browser app using js-ipfs which allows you to view (and potentially interact/manipulate in future) git repositories.

• go-ipfs repository -> https://igis.devtty.eu/#/repo/z8mWaFhNutrvGaKNcybtLjgLMEC3n5tC5 (igis.devtty.eu can be accessed as /ipns/igis.devtty.eu too)
• Source -> https://github.com/ipfs-shipyard/IGiS

Note that js-ipfs may not be able to resolve your repositories unless you connect directly to your node (this will soon not be needed as dht/relay js-ipfs features are developed). You can do that in the app with ipfs.node$1.swarm.connect('/your/multiaddr/ws/address')..

[pruflyos]

There is no direct mapping between the two - you need a separate viewer.

So a workaround/process would be:

1. checkout the head revision of an ipld repository
2. ipfs add README.md

And share that hash. I just wonder if the same data is then twice in IPFS?

[magik6k]

I just wonder if the same data is then twice in IPFS?

Unfortunately yes - deduplicating data between different formats would require hashing the data with all supported hash algorithms, checking for duplicates in the ipfs datastore, and then somehow linking the data together. Doing this would be really expensive cpu/io wise.

[pruflyos]

deduplicating data between different formats...

OK. It was not clear to me, that the data pushed to git is in a different format then it is "normally"...

[pruflyos]

One more question:

Assume I pull/push on a workstation which is not 24/7 "online".

1. I assume that all data in the ipld repository is "pinned" automatically and never garbage collected!?
2. Is there any way that I can push (spread) the ipld data automatically to another node which is online 24/7?

So that when I push (git push) something to IPFS on my local workstation, it is additionally available at least on one 24/7 node in the IPFS network? It doesn't need to be a random node, would be fine if I could explicitly define another node I control.

[magik6k]

I assume that all data in the ipld repository is "pinned" automatically and never garbage collected

It's not, dag put (which is used internally here), doesn't pin by-default.

Is there any way that I can push (spread) the ipld data automatically to another node which is online 24/7?

When you push you get a line like this:

Head CID is z8mWaFiA3ZiXGAPg8vKUJpnsGMErzLyN5

z8mW... is CIDv1 of the HEAD (usually the latest commit). You can pin that on another nodes with the usual ipfs pin add z8m.... Again, this is something that will get way easier/less annoying with the ipns helper and when this go-ipfs PR is merged.

[magik6k]

You can also create git CIDv1 from sha1 hash with this bash alias/function:

alias gitcid='f() { printf z; printf "01781114$1" | bases hex base58; echo };f'

For the bases program just do go get github.com/whyrusleeping/bases

[pruflyos]

You can also create git CIDv1 from sha1 hash with this bash alias/function:

Which sha1? Of the last file I pushed?
sha1sum file | gitcid ? What params should I call gitcid with?

[magik6k]

You call it with sha1 of latest commit - gitcid 2347e110c29742a1783134ef45f5bff58b29e40e

[pruflyos]

You call it with sha1 of latest commit - gitcid 2347e110c29742a1783134ef45f5bff58b29e40e

Then there must be something wrong with the alias function you gave me above. When I call it, it just keeps asking for input until I hit Ctrl-c.