Publishing websites to IPFS as a service and github app

[olizilla]

ETA April 2020:

Valuable discussion on this topic has also been gathered in #69, which was closed in favor of this issue. Please consider discussion in that issue when proceeding with this one.

Original issue text

The problem: I want to publish a website to IPFS and also my domain, with https, and not have it rely (just) on my machine for uptime. We want shareable preview urls, and a simple mechanism for promoting a preview to the live verison.

Prior art:

• surge.sh - set the bar for simple website publishing... just run surge from the directory you want to publish
• now.sh - introduced us to the idea of immutable deploys, but publishing each

How about "pin sites to an ipfs cluster, as a service, and github app"

Example usage

ipfsify ./build
Adding to IPFS...
Pinning on cluster.ipfs.dev...
Preview ready https://bafywoowoo.ipfs.dev

The command would add a site locally, and pin the CID to the remote cluster. You get an immutable deploy style url like https://bafywoowoo.ipfs.dev that loads fast as the content is already on more than one machine, and one of them is an always available cluster that we look after.

...some preview builds later...

me: Ok that CID is hot! I wish to publish it!

bot: OK, as this a new site, let me create a IPNS keypair for you. Here it is, dont lose it. I'm going to host it for you and point it to that new CID you gave me, and reliably reannounce it every few hours for you... do you have a legacy dns name you'd like to map it to?

me: sure do, it's oli.zilla.org.uk, what of it?

bot: ok, go set this txt record on your dns provider dnslink=/ipns/Quink...

me: done!

bot: ok, performing the lets encrypt dance... done. https://oli.zilla.org.uk is go, dweb style.

---

This need has come up multiple times, from mutliple folks. Right now there are a bunch of PL sites that need this service. We've got a reasonable service in place via circleci, but the drawbacks are

• using the main gateway for hosting. we dont want to add load to them for every site we host
• using the main cluster for pinning. This use-case can use its own cluster.
• https config is manual
• we have to update an ipfs dnslink via a dnsimple token for every build we want to promote to live... why not just use ipns.
• we can't enable the circleci builds for PRs from forks as we'd risk exposing our dns token. This sucks as we want to enable this process for everyone.

An extension to this idea is to have a github action that builds, pins and updates the status on your PR with the preview url (done, here: https://github.com/ipfs-shipyard/ipfs-github-action)
and a github app that is notified when your master branch changes, and triggers the ipns update automagically.

[cwaring]

+1 to more conversational issues 😂

@scout and I were talking about this same idea/service just a couple of days ago. I'm very interested in sketching out thoughts to make this happen. I have a few concerns around using dns to link to deployed versions, due to browsers not always respecting TTL but this is something i need to get a better feel for first.

I'd love to tag team on this next Wed!

[olizilla]

Hurray!

The trick to this proposal is that we want to stop having to mess with DNS to update a site. We really want to be able to just ipfs name publish --key=oli.zilla.org.uk <bafywoowoo> to update the IPNS pointer to the latest CID. The reason we haven't done so in the past is due to IPNS resolution speed, but i want to test that out too, as we just need that to work and be fast.

[scout]

Count me in!

[jimpick]

This is cool. I want it now! :-)

(Fun fact - At Joyent, I used to work with Brock and Rob who made surge.sh ... they also created PhoneGap)

[agentofuser]

I'd love to help with this any way I can.

I actually already have a cli working pretty much as @olizilla described: https://github.com/agentofuser/ipfs-deploy.

Next up is making it work with Travis so that it automatically deploys PRs and branches, and posts the link back as a comment.

Besides Travis I also want to make it work with as many CIs as possible like Gitlab CI, Circle CI, etc.

People have reacted very positively to my pitch of "IPFS as easy as Netlify" :)

[agentofuser]

I'd love to help with this any way I can.

I actually already have a cli working pretty much as @olizilla described: https://github.com/agentofuser/ipfs-deploy.

Next up is making it work with Travis so that it automatically deploys PRs and branches, and posts the link back as a comment.

Besides Travis I also want to make it work with as many CIs as possible like Gitlab CI, Circle CI, etc.

People have reacted very positively to my pitch of "IPFS as easy as Netlify" :)

[olizilla]

@agentofuser you did it! This is very cool! I'm happy to help out how ever I can!

[jessicaschilling]

NOTE: Valuable discussion on this topic has also been gathered in #69, which was closed in favor of this issue. Please consider discussion in that issue when proceeding with this one.

[jessicaschilling]

Note ipfs/ipfs-webui#1441 - user request for this functionality.

[jessicaschilling]

Note ipfs/ipfs-webui#1453 - user request for this functionality.

[jessicaschilling]

Other inspiration per @lidel - Beaker Browser, https://beakerbrowser.com/

[lidel]

I believe this will become much more relevant when "pinning API changes" land in WebUI.
People will be able to easily pin website data to remote nodes to keep them available when their laptop/pc is off.

After that lands, we should look into ways WebUI can provide basic Publishing tools, so people can edit and publish static websites to IPFS without leaving WebUI etc. Editor UIs in Beaker and #69 provide good starting points for exploration of the "publishing from webui" angle.

[jessicaschilling]

Referencing ipfs/ipfs-webui#1696 (comment) as another consideration, to quote:

Whenever you generate a new key with ipfs key gen --type=rsa --size=2048 peername, it seems messy to set up a folder in the self peer to assign to the new peer.

My idea to manage this is to have a dropdown selection on the PEER ID entry in the STATUS table.
The contents of the dropdown selection would be the keys from ipfs key list

What I expect here is selecting an alternate key and then clicking the FILES tab will allow you to manage the files of the selected peer.

EDIT: Since the current setup relies on folders, integrating this could provide an alternate option to automatically create a static root folder under the new peer name within the self peer.
(the folder could be highlighted red to denote it as a sub-peer)
Being an option, this could be disabled for the current behavior.