IPNS: should work with case-insensitive identifiers (Base32) #5287

lidel · 2018-07-26T14:17:16Z

Version information:

0.4.16

Type:

bug

Description:

During the work related to ipfs/in-web-browsers#89 and https://github.com/ipfs/ipfs/issues/337 we've identified open problem with IPNS. We are unable to use current case-sensitive IPNS identifiers in case-insensitive URL contexts, namely:

https://<keyId>.ipns.foo.tld/
ipns://<keyId>/

keyId – PeerID or ID of a key generated via ipfs key gen

Not sure what would be the best way to tackle this.
Adding only a multibase prefix? Switching all keyId to CIDv1 ?

Part of ipfs/ipfs#337 (cc @kyledrake + @alanshaw for parity in js-ipfs)

Additional References:

2018-Q4: Ethereum people started using IPFS+ENS for real, MetaMask supports resolver for hosting *.eth websites and they just noticed the Origin problem.
- The fix is cid-in-subdomain (CID as a Subdomain in-web-browsers#89) but it only works for /ipfs/ and at some point they will want to point at /ipns/ instead of /ipfs/, which is blocked by this issue
2019-10: Tracking issue for peer IDs as cidv1b32: Tracking issue for peer IDs as CIDs libp2p/specs#216

The text was updated successfully, but these errors were encountered:

ghost · 2018-07-26T14:18:58Z

I think we'll have to wait for ed25519 IPNS keys, or do the same hackery as with CIDs.

Stebalien · 2018-07-26T17:18:37Z

ed25519 IPNS keys are here. However, that doesn't help. We'll have to do the same CID hackery (or just switch to using CIDs).

Missed the fact IPNS does not work in case-insensitive contexts right now. More: ipfs/kubo#5287

ghost · 2018-09-20T20:41:05Z

Let's switch to CIDs?

@kyledrake this issue should potentially be part of the base32 task force

lidel · 2019-03-18T17:13:50Z

Related discussion about PeerID in libp2p continues in libp2p/specs#138

lidel · 2019-04-15T18:02:50Z

I believe the MVP here is a scenario where:

people convert old Qm.. PeerID (base58btc - cidv0 - dag-pb - sha2-256)
to bafy.. (base32 - cidv1 - dag-pb - sha2-256)
- (via http://cid.ipfs.io or ipfs cid base32 command)
use the base32 result in /ipns/bafy.. addresses
.. and it just works (tm)

@Stebalien What would be the smallest set of changes to support this type of interop in go-ipfs?
Is it something someone not deeply familiar with go-ipfs codebase could PR?
If so, can you give me some pointers where to start?

Stebalien · 2019-04-17T17:58:34Z

The current peer ID format is just base58btc(<sha2-256 multicodec><256><digest>) (just a multihash).
The new format would be <base32 multibase> ++ base32(<cidv1><libp2p key multicodec><the multihash>).

That is, we'd use an entirely new multicodec.

@Stebalien What would be the smallest set of changes to support this type of interop in go-ipfs?

That depends on how deeply we want to integrate this. If we only care about this for the subdomain feature, it would be pretty easy to redirect GATEWAY/ipns/Qm... to b...ipns.GATEWAY/.

lidel · 2019-04-17T23:41:29Z

There is an ongoing work to get IPNS fast (notes from today's meeting: ipfs/team-mgmt#948).
In the short term, fast resolve of IPNS in web browser context will most likely mean custom DNS server and resolving IPNS over DNS TXT records looking like this: dnslink=/ipns/<bafy>.ipns.dweb.link (where <bafy> is the new format in base32)

iiuc this means the immediate needs are:

Agree and add <libp2p key multicodec> to multicodec/table.csv and related libs, to unlock IPNS representation in subdomains (for actual hosting, and IPNS-over-DNS lookups)
- Q: will this be just a single multicodec, or a family of entries depending on key subtype? are there any prior proposals on this?
Add support to HTTP Gateway for requests for <bafy>.ipns.GATEWAY/ and /ipns/<bafy>
- Q: I imagine Gateway could just convert <bafy> to old school base58btc before doing the actual resolve, but it may get confusing if ipfs name resolve /ipns/<bafy> does not work. Would it be feasible to do the conversion there as well?

lidel · 2019-05-14T09:35:47Z

I see a small UX problem around having a dedicated multicodec:
How will go-ipfs behave if codec in /ipns/{cidv1} is different than libp2p-key?

go-ipfs prints PeerID in Base58 QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN
User wants to convert it to CIDv1, so they do ipfs cid base32 {cid} or use https://cid.ipfs.io/
cid base32 returns bafybeiagwnqiviaae5aet2zivwhhsorg75x2wka2pu55o7grr23ulx5kxm which has the dag-pb codec instead of libp2p-key (multicodec: add IPLD codec for libp2p public keys multiformats/multicodec#131)

This issue will mostly go away when we switch to displaying PeerID in CIDv1 everywhere, but for now we need to account for the above edge case.
I believe we should support support PeerID CIDv1 with dag-pb. If that is not feasible, then the gateway should do conversion and return HTTP 301 redirect from dag-pg PeerID to one with libp2p-key.

@Stebalien Thoughts?

Zolmeister · 2019-05-25T22:21:53Z

Please consider using Tor rend spec v3 address encoding for PeerIDs
Ref. libp2p/specs#139

lidel · 2019-05-27T23:42:28Z

@Zolmeister thank you for sharing. This issue is only about adding support for wrapping existing PeerIDs in CIDv1 to unlock web uses without huge refactoring (it does not change underlying way we construct PeerID). Discussion in libp2p/specs#139 is not blocked by this, feel free to continue it there 👍

lidel · 2019-05-27T23:44:54Z

Back to the issue at hand:

Adding IPLD codec for libp2p public keys (multicodec: add IPLD codec for libp2p public keys multiformats/multicodec#131) got merged
I opened feat: add libp2p-key and garlic32 to js-multicodec multiformats/js-multicodec#45 and Add support for libp2p-key multicodec to go-cid go-cid#86 to add support for it in GO and JS
- I assumed go-cid is the right place, as go-multicodec is deprecated
- When this lands, we will add support for CIDv1 in /ipns/ paths to go-ipfs/js-ipfs/is-ipfs and it should be it.

lidel · 2019-06-03T17:23:40Z

I've been looking at things we want to ship this in time for IPFS Camp.
This one looks doable. What remains here is:

add libp2p-key support to Add support for libp2p-key multicodec to go-cid go-cid#86
release new go-cid with Add support for libp2p-key multicodec to go-cid go-cid#86
update go-ipfs to use new go-cid

@Stebalien will this be enough for HTTP Gateway support /ipns/<cidv1-with-libp2p-key>? If not, can you point me at places that need tweaking?

Stebalien · 2019-07-24T21:25:21Z

#5287 (comment)

Sorry, completely missed this conversation. That's a valid concern and we may want to add a "hack it" redirect for now, or maybe a warning?

@Stebalien will this be enough for HTTP Gateway support /ipns/? If not, can you point me at places that need tweaking?

Bit late but the latest status is libp2p/go-libp2p-core#41

lidel · 2019-07-29T10:35:44Z

How will go-ipfs behave if codec in /ipns/{cidv1} is different than libp2p-key?

That's a valid concern and we may want to add a "hack it" redirect for now, or maybe a warning?

I am thinking the least amount of work/friction will be to

Do HTTP 301 redirect on the Gateway to fix the multicodec on-the-fly.
This will ensure user ends up on a valid, canonical address in the browser.
"just work" on the CLI
(ignore invalid multicodec on /ipns/{cid}, or display a warning with a valid path ready to copy&paste)

The issue will go away when we switch to CIDv1B32 everywhere,
so we should not stress too much about beyond making it work.

Stebalien · 2019-08-08T01:35:37Z

Yeah, I guess. Just... grrr.

lidel · 2019-10-01T18:56:28Z

Tracking issue for peer IDs as cidv1b32: libp2p/specs#216

lidel · 2020-02-05T23:16:27Z

We are in the process of adding support for PeerID encoded as CIDv1 with libp2p-key multihash (libp2p/specs#216).

Seems that even tho we use go-libp2p-core v0.3.0 which includes libp2p/go-libp2p-core#41 there are still places which assume Multihash instead of CID:

$ ipfs name resolve -r /ipns/bafzbeiedwhhfhtte23hocn4k2bi4awpqay5sdhzqju7htk3y6sk63lvuae
Error: not a valid proquint string

$ ipfs resolve -r /ipns/bafzbeiedwhhfhtte23hocn4k2bi4awpqay5sdhzqju7htk3y6sk63lvuae
Error: not a valid proquint string

I'll try to track them down and fix, as it is blocking me from writing tests for #6096 😬

Update: found, will fix in #6096

This adds support for resolving /ipns/<cidv1-libp2p-key> cc #5287 License: MIT Signed-off-by: Marcin Rataj <lidel@lidel.org>

lidel · 2020-04-16T15:18:41Z

This was fixed in #6096 and will ship with go-ipfs 0.5.0 (#7109) 🚀

When used in subdomain context, the HTTP Gateway will automatically convert case-sensitive identifier to base32 and redirect to URL with correct multicodec.

Command line and RPC API provide handy error message when an invalid multicodec is used in CIDv1:

# resolve Base58 PeerID
$ ipfs name resolve -r /ipns/QmY9DNm4UJxaTBB9FrhNkXQ75GaK43D8KhZtpJjmw8P7Sz
/ipfs/bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi

# convert PeerID to CIDv1 (with invalid multicoded, to illustrate next step)
$ ipfs cid base32 QmY9DNm4UJxaTBB9FrhNkXQ75GaK43D8KhZtpJjmw8P7Sz
bafybeieru3k6wvdzsrsjcx33jund7ebawoqlyc7jpitmn7wd4cwveunolm

# resolve CIDv1 with dag-pb multicodec
$ ipfs name resolve -r /ipns/bafybeieru3k6wvdzsrsjcx33jund7ebawoqlyc7jpitmn7wd4cwveunolm
Error: peer ID represented as CIDv1 require libp2p-key multicodec: retry with /ipns/bafzbeieru3k6wvdzsrsjcx33jund7ebawoqlyc7jpitmn7wd4cwveunolm

# resolve CIDv1 with libp2p-key multicodec suggested in previous step
$ ipfs name resolve -r /ipns/bafzbeieru3k6wvdzsrsjcx33jund7ebawoqlyc7jpitmn7wd4cwveunolm
/ipfs/bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi

lidel added a commit to ipfs/in-web-browsers that referenced this issue Jul 26, 2018

docs: removed IPNS in subdomain

826328e

Missed the fact IPNS does not work in case-insensitive contexts right now. More: ipfs/kubo#5287

This was referenced Aug 22, 2018

Suborigins ipfs/in-web-browsers#66

Closed

Suborigin header must be lowercase #3209

Closed

lidel mentioned this issue Sep 17, 2018

[RFC] peer discovery with mDNS libp2p/specs#80

Merged

lidel mentioned this issue Sep 22, 2018

ipfs peer support ipfs-shipyard/is-ipfs#15

Closed

lidel mentioned this issue Oct 18, 2018

IPFS Addressing in Web Browsers, ipns://, and DNSLink ipfs/in-web-browsers#118

Open

lidel mentioned this issue Nov 14, 2018

Specify peer ID hash function in public key libp2p/specs#111

Closed

Zolmeister mentioned this issue Jan 18, 2019

Multibase PeerIDs #5589

Closed

lidel mentioned this issue Mar 29, 2019

Redirect of CID-in-subdomain and DNSLink websites ipfs/ipfs-companion#667

Closed

hugomrdias mentioned this issue Apr 18, 2019

IPNS over DNS ipfs/js-ipfs#2000

Closed

15 tasks

lidel mentioned this issue May 13, 2019

Add multicodec for libp2p key multiformats/multicodec#130

Closed

This was referenced May 27, 2019

feat: add libp2p-key and garlic32 to js-multicodec multiformats/js-multicodec#45

Merged

Add support for libp2p-key multicodec to go-cid ipfs/go-cid#86

Merged

lidel mentioned this issue May 28, 2019

post: add JS IPFS 0.36 release post ipfs-inactive/blog#244

Merged

2 tasks

lidel mentioned this issue Jun 10, 2019

Add examples to ADDRESSING.md ipfs/in-web-browsers#148

Merged

lidel mentioned this issue Jun 19, 2019

CID as a Subdomain ipfs/in-web-browsers#89

Open

This was referenced Aug 19, 2019

fix: add support for libp2p-key multicodec multiformats/cid-utils-website#14

Merged

RFC 0001: Text Peer Ids as CIDs libp2p/specs#209

Merged

lidel mentioned this issue Aug 29, 2019

fix: add robots.txt to exclude gateway paths ipfs-inactive/website#330

Closed

This was referenced Oct 25, 2019

feat: support Peer ID represented as CID libp2p/js-peer-id#105

Merged

feat: js-ipfs support of CIDs in /ipns/ content paths ipfs/js-ipfs#2566

Merged

Stebalien mentioned this issue Feb 5, 2020

Tracking issue for peer IDs as CIDs libp2p/specs#216

Open

22 tasks

lidel added a commit that referenced this issue Feb 6, 2020

feat: resolve peer IDs as CIDs

77b8bd4

This adds support for resolving /ipns/<cidv1-libp2p-key> cc #5287 License: MIT Signed-off-by: Marcin Rataj <lidel@lidel.org>

lidel added a commit that referenced this issue Feb 11, 2020

feat: resolve peer IDs as CIDs

ef00a67

This adds support for resolving /ipns/<cidv1-libp2p-key> cc #5287 License: MIT Signed-off-by: Marcin Rataj <lidel@lidel.org>

lidel added a commit that referenced this issue Mar 4, 2020

feat: resolve peer IDs as CIDs

5ca5985

This adds support for resolving /ipns/<cidv1-libp2p-key> cc #5287 License: MIT Signed-off-by: Marcin Rataj <lidel@lidel.org>

lidel mentioned this issue Apr 1, 2020

Move IPFS addressing spec to ipfs/specs ipfs/specs#248

Open

14 tasks

lidel closed this as completed Apr 16, 2020

lidel mentioned this issue Apr 16, 2020

Migration to CIDv1 (default base32) ipfs/specs#247

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IPNS: should work with case-insensitive identifiers (Base32) #5287

IPNS: should work with case-insensitive identifiers (Base32) #5287

lidel commented Jul 26, 2018 •

edited

Loading

ghost commented Jul 26, 2018

Stebalien commented Jul 26, 2018

ghost commented Sep 20, 2018 •

edited by ghost

Loading

lidel commented Mar 18, 2019

lidel commented Apr 15, 2019 •

edited

Loading

Stebalien commented Apr 17, 2019

lidel commented Apr 17, 2019 •

edited

Loading

lidel commented May 14, 2019 •

edited

Loading

Zolmeister commented May 25, 2019

lidel commented May 27, 2019

lidel commented May 27, 2019 •

edited

Loading

lidel commented Jun 3, 2019 •

edited

Loading

Stebalien commented Jul 24, 2019

lidel commented Jul 29, 2019 •

edited

Loading

Stebalien commented Aug 8, 2019

lidel commented Oct 1, 2019

lidel commented Feb 5, 2020 •

edited

Loading

lidel commented Apr 16, 2020 •

edited

Loading

IPNS: should work with case-insensitive identifiers (Base32) #5287

IPNS: should work with case-insensitive identifiers (Base32) #5287

Comments

lidel commented Jul 26, 2018 • edited Loading

Version information:

Type:

Description:

Additional References:

ghost commented Jul 26, 2018

Stebalien commented Jul 26, 2018

ghost commented Sep 20, 2018 • edited by ghost Loading

lidel commented Mar 18, 2019

lidel commented Apr 15, 2019 • edited Loading

Stebalien commented Apr 17, 2019

lidel commented Apr 17, 2019 • edited Loading

lidel commented May 14, 2019 • edited Loading

Zolmeister commented May 25, 2019

lidel commented May 27, 2019

lidel commented May 27, 2019 • edited Loading

lidel commented Jun 3, 2019 • edited Loading

Stebalien commented Jul 24, 2019

lidel commented Jul 29, 2019 • edited Loading

Stebalien commented Aug 8, 2019

lidel commented Oct 1, 2019

lidel commented Feb 5, 2020 • edited Loading

lidel commented Apr 16, 2020 • edited Loading

lidel commented Jul 26, 2018 •

edited

Loading

ghost commented Sep 20, 2018 •

edited by ghost

Loading

lidel commented Apr 15, 2019 •

edited

Loading

lidel commented Apr 17, 2019 •

edited

Loading

lidel commented May 14, 2019 •

edited

Loading

lidel commented May 27, 2019 •

edited

Loading

lidel commented Jun 3, 2019 •

edited

Loading

lidel commented Jul 29, 2019 •

edited

Loading

lidel commented Feb 5, 2020 •

edited

Loading

lidel commented Apr 16, 2020 •

edited

Loading