Skip to content

v0.10.0
Compare
Choose a tag to compare
@aschmahmann aschmahmann released this 01 Oct 19:23
· 2589 commits to master since this release
v0.10.0
This tag was signed with the committer’s verified signature.
aschmahmann Adin Schmahmann
GPG key ID: D7A248B705EE4F69
Learn about vigilant mode.
64b532f
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

go-ipfs v0.10.0 Release

We're happy to announce go-ipfs 0.10.0. This release brings some big changes to the IPLD internals of go-ipfs that make working with non-UnixFS DAGs easier than ever. There are also a variety of new commands and configuration options available.

As usual, this release includes important fixes, some of which may be critical for security. Unless the fix addresses a bug being exploited in the wild, the fix will not be called out in the release notes. Please make sure to update ASAP. See our release process for details.

🛠 TLDR: BREAKING CHANGES

  • ipfs dag get
    • default output changed to dag-json
    • dag-pb (e.g. unixfs) field names changed - impacts userland code that works with dag-pb objects returned by dag get
    • no longer emits an additional new-line character at the end of the data output
  • ipfs dag put
    • defaults changed to reduce ambiguity and surprises: input is now assumed to be dag-json, and data is serialized to dag-cbor at rest.
    • --format and --input-enc were removed and replaced with --store-codec and --input-codec
    • codec names now match the ones defined in the multicodec table
    • dag-pb (e.g. unixfs) field names changed - impacts userland code that works with dag-pb objects stored via dag put

Keep reading to learn more details.

🔦 Highlights

🌲 IPLD Levels Up

The handling of data serialization as well as many aspects of DAG traversal and pathing have been migrated from older libraries, including go-merkledag and go-ipld-format to the new go-ipld-prime library and its components. This allows us to use many of the newer tools afforded by go-ipld-prime, stricter and more uniform codec implementations, support for additional (pluggable) codecs, and some minor performance improvements.

This is significant refactor of a core component that touches many parts of IPFS, and does come with some breaking changes:

  • IPLD plugins:
    • The PluginIPLD interface has been changed to utilize go-ipld-prime. There is a demonstration of the change in the bundled git plugin.
  • The semantics of dag put and dag get change:
    • dag get now takes the output-codec option which accepts a multicodec name used to encode the output. By default this is dag-json, which is a strict and deterministic subset of JSON created by the IPLD team. Users may notice differences from the previously plain Go JSON output, particularly where bytes are concerned which are now encoded using a form similar to CIDs: {"/":{"bytes":"unpadded-base64-bytes"}} rather than the previously Go-specific plain padded base64 string. See the dag-json specification for an explanation of these forms.
    • dag get no longer prints an additional new-line character at the end of the encoded block output. This means that the output as presented by dag get are the exact bytes of the requested node. A round-trip of such bytes back in through dag put using the same codec should result in the same CID.
    • dag put uses the input-codec option to specify the multicodec name of the format data is being provided in, and the store-codec option to specify the multicodec name of the format the data should be stored in at rest. These formerly defaulted to json and cbor respectively. They now default to dag-json and dag-cbor respectively but may be changed to any supported codec (bundled or loaded via plugin) by its multicodec name.
    • The json and cbor multicodec names (as used by input-enc and format options) are now no longer aliases for dag-json and dag-cbor respectively. Instead, they now refer to their proper multicodec types. cbor refers to a plain CBOR format, which will not encode CIDs and does not have strict deterministic encoding rules. json is a plain JSON format, which also won't encode CIDs and will encode bytes in the Go-specific padded base64 string format rather than the dag-json method of byte encoding. See https://ipld.io/specs/codecs/ for more information on IPLD codecs.
    • protobuf is no longer used as the codec name for dag-pb
    • The codec name raw is used to mean Bytes in the IPLD Data Model
  • UnixFS refactor. The dag-pb codec, which is used to encode UnixFS data for IPFS, is now represented through the dag API in a form that mirrors the protobuf schema used to define the binary format. This unifies the implementations and specification of dag-pb across the IPLD and IPFS stacks. Previously, additional layers of code for file and directory handling within IPFS between protobuf serialization and UnixFS obscured the protobuf representation. Much of this code has now been replaced and there are fewer layers of transformation. This means that interacting with dag-pb data via the dag API will use different forms:
    • Previously, using dag get on a dag-pb block would present the block serialized as JSON as {"data":"padded-base64-bytes","links":[{"Name":"foo","Size":100,"Cid":{"/":"Qm..."}},...]}.
    • Now, the dag-pb data with dag-json codec for output will be serialized using the data model from the dag-pb specification: {"Data":{"/":{"bytes":"unpadded-base64-bytes"}},"Links":[{"Name":"foo","Tsize":100,"Hash":{"/":"Qm..."}},...]}. Aside from the change in byte formatting, most field names have changed: data → Data, links → Links, Size → Tsize, Cid → Hash. Note that this output can be changed now using the output-codec option to specify an alternative codec.
    • Similarly, using dag put and a store-codec option of dag-pb now requires that the input conform to this dag-pb specified form. Previously, input using {"data":"...","links":[...]} was accepted, now it must be {"Data":"...","Links":[...]}.
    • Previously it was not possible to use paths to navigate to any of these properties of a dag-pb node, the only possible paths were named links, e.g. dag get QmFoo/NamedLink where NamedLink was one of the links whose name was NamedLink. This functionality remains the same, but by prefixing the path with /ipld/ we enter data model pathing semantics and can dag get /ipld/QmFoo/Links/0/Hash to navigate to links or /ipld/QmFoo/Data to simply retrieve the data section of the node, for example.
    • ℹ See the dag-pb specification for details on the codec and its data model representation.
    • ℹ See this detailed write-up for further background on these changes.

Ⓜ Multibase Command

go-ipfs now provides utility commands for working with multibase:

$ echo -n hello | ipfs multibase encode -b base16 > file-mbase16
$ cat file-mbase16
f68656c6c6f

$ ipfs multibase decode file-mbase16
hello

$ cat file-mbase16 | ipfs multibase decode
hello

$ ipfs multibase transcode -b base2 file-mbase16
00110100001100101011011000110110001101111

See ipfs multibase --help for more examples.

🔨 Bitswap now supports greater configurability

This release adds an Internal section to the configuration file that is designed to help advanced users optimize their setups without needing a custom binary. The Internal section is not guaranteed to be the same from release to release and may not be covered by migrations. If you use the Internal section you should be making sure to check the config documentation between releases for any changes.

🐚 Programmatic shell completions command

ipfs commands completion bash will generate a bash completion script for go-ipfs commands

📜 Profile collection command

Performance profiles can now be collected using ipfs diag profile. If you need to do some debugging or have an issue to submit the collected profiles are very useful to have around.

🍎 Mac OS notarized binaries

The go-ipfs and related migration binaries (for both Intel and Apple Sillicon) are now signed and notarized to make Mac OS installation easier.

👨‍👩‍👦 Improved MDNS

There is a completed implementation of the revised libp2p MDNS spec. This should result in better MDNS discovery and better local/offline operation as a result.

🚗 CAR import statistics

dag import command now supports --stats option which will include the number of imported blocks and their total size in the output.

🕸 Peering command

This release adds swarm peering command for easy management of the peering subsystem. Peer in the peering subsystem is maintained to be connected at all times, and gets reconnected on disconnect with a back-off.

See ipfs swarm peering --help for more details.

Changelog

Full Changelog

Contributors

Contributor Commits Lines ± Files Changed
Daniel Martí 42 +8549/-6587 170
Eric Myhre 55 +5883/-6715 395
Marten Seemann 100 +1814/-2028 275
Steven Allen 80 +1573/-1998 127
hannahhoward 18 +1721/-671 53
Will 2 +1114/-1217 18
Andrew Gillis 2 +1220/-720 14
gammazero 3 +43/-1856 10
Masih H. Derkani 3 +960/-896 8
Adin Schmahmann 25 +1458/-313 44
vyzo 27 +986/-353 60
Will Scott 6 +852/-424 16
Rod Vagg 19 +983/-255 66
Petar Maymounkov 6 +463/-179 22
web3-bot 10 +211/-195 24
adlrocha 1 +330/-75 15
RubenKelevra 2 +128/-210 2
Ian Davis 3 +200/-109 17
Cory Schwartz 3 +231/-33 7
Keenan Nemetz 1 +184/-71 2
Randy Reddig 2 +187/-53 8
Takashi Matsuda 3 +201/-2 7
guseggert 4 +161/-20 9
Lucas Molas 5 +114/-47 27
nisdas 4 +115/-45 7
Michael Muré 6 +107/-33 24
Richard Ramos 2 +113/-9 3
Marcin Rataj 12 +88/-24 13
Ondrej Prazak 2 +104/-6 4
Michal Dobaczewski 2 +77/-28 3
Jorropo 3 +9/-75 4
Andey Robins 1 +70/-3 3
Gus Eggert 10 +34/-31 12
noot 1 +54/-9 5
Maxim Merzhanov 1 +29/-24 1
Adrian Lanzafame 1 +30/-13 2
Bogdan Stirbat 1 +22/-16 2
Shad Sterling 1 +28/-3 1
Jesse Bouwman 5 +30/-0 5
Pavel Karpy 1 +19/-7 2
lasiar 5 +14/-10 5
Dennis Trautwein 1 +20/-4 2
Louis Thibault 1 +22/-1 2
whyrusleeping 2 +21/-1 2
aarshkshah1992 3 +12/-8 3
Peter Rabbitson 2 +20/-0 2
bt90 2 +17/-2 2
Dominic Della Valle 1 +13/-1 2
Audrius Butkevicius 1 +12/-1 1
Brian Strauch 1 +9/-3 1
Aarsh Shah 2 +1/-11 2
Whyrusleeping 1 +11/-0 1
Max 1 +7/-3 1
vallder 1 +3/-5 1
Michael Burns 3 +2/-6 3
Lasse Johnsen 1 +4/-4 2
snyh 1 +5/-2 1
Hector Sanjuan 2 +3/-2 2
市川恭佑 (ebi) 1 +1/-3 1
godcong 2 +2/-1 2
Mathis Engelbart 1 +1/-2 1
folbrich 1 +1/-1 1
Med Mouine 1 +1/-1 1
Assets 50
Loading