Skip to content

deps!: update multiformats to 10.x.x and all @ipld/* modules #265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 19, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# js-ipfs-unixfs <!-- omit in toc -->

[![ipfs.io](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io)
[![IRC](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
[![Discord](https://img.shields.io/discord/806902334369824788?style=flat-square)](https://discord.gg/ipfs)
[![ipfs.tech](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](https://ipfs.tech)
[![Discuss](https://img.shields.io/discourse/https/discuss.ipfs.tech/posts.svg?style=flat-square)](https://discuss.ipfs.tech)
[![codecov](https://img.shields.io/codecov/c/github/ipfs/js-ipfs-unixfs.svg?style=flat-square)](https://codecov.io/gh/ipfs/js-ipfs-unixfs)
[![CI](https://img.shields.io/github/workflow/status/ipfs/js-ipfs-unixfs/test%20&%20maybe%20release/master?style=flat-square)](https://github.com/ipfs/js-ipfs-unixfs/actions/workflows/js-test-and-release.yml)

Expand Down Expand Up @@ -74,8 +73,12 @@ Licensed under either of

## Contribute

Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-importer/issues)!
Contributions welcome! Please check out [the issues](https://github.com/ipfs/js-ipfs-unixfs/issues).

This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
Also see our [contributing document](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md) for more information on how we work, and about contributing in general.

Please be aware that all interactions related to this repo are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

[![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)
7 changes: 5 additions & 2 deletions lerna.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
{
"lerna": "4.0.0",
"lerna": "5.4.0",
"useWorkspaces": true,
"version": "independent",
"command": {
"run": {
"stream": true
}
}
},
"packages": [
"packages/*"
]
}
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
"release": "lerna exec --concurrency 1 -- semantic-release -e semantic-release-monorepo"
},
"dependencies": {
"lerna": "^5.0.0"
"lerna": "^6.0.1"
},
"workspaces": [
"packages/*"
Expand Down
25 changes: 14 additions & 11 deletions packages/ipfs-unixfs-exporter/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# ipfs-unixfs-exporter <!-- omit in toc -->

[![ipfs.io](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io)
[![IRC](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
[![Discord](https://img.shields.io/discord/806902334369824788?style=flat-square)](https://discord.gg/ipfs)
[![ipfs.tech](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](https://ipfs.tech)
[![Discuss](https://img.shields.io/discourse/https/discuss.ipfs.tech/posts.svg?style=flat-square)](https://discuss.ipfs.tech)
[![codecov](https://img.shields.io/codecov/c/github/ipfs/js-ipfs-unixfs.svg?style=flat-square)](https://codecov.io/gh/ipfs/js-ipfs-unixfs)
[![CI](https://img.shields.io/github/workflow/status/ipfs/js-ipfs-unixfs/test%20&%20maybe%20release/master?style=flat-square)](https://github.com/ipfs/js-ipfs-unixfs/actions/workflows/js-test-and-release.yml)

Expand Down Expand Up @@ -236,12 +235,6 @@ for await (const child of recursive('Qmfoo/foo/bar', blockstore)) {
// entries contains all children of the `Qmfoo/foo/bar` directory and it's children
```

[dag API]: https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/DAG.md

[blockstore]: https://github.com/ipfs/js-ipfs-interfaces/tree/master/packages/interface-blockstore#readme

[UnixFS]: https://github.com/ipfs/specs/tree/master/unixfs

## Contribute

Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-exporter/issues)!
Expand All @@ -259,8 +252,18 @@ Licensed under either of

## Contribute

Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-importer/issues)!
Contributions welcome! Please check out [the issues](https://github.com/ipfs/js-ipfs-unixfs/issues).

This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
Also see our [contributing document](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md) for more information on how we work, and about contributing in general.

Please be aware that all interactions related to this repo are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

[![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)

[dag API]: https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/DAG.md

[blockstore]: https://github.com/ipfs/js-ipfs-interfaces/tree/master/packages/interface-blockstore#readme

[UnixFS]: https://github.com/ipfs/specs/tree/master/unixfs
22 changes: 11 additions & 11 deletions packages/ipfs-unixfs-exporter/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -151,21 +151,21 @@
"release": "aegir release"
},
"dependencies": {
"@ipld/dag-cbor": "^7.0.2",
"@ipld/dag-pb": "^2.0.2",
"@multiformats/murmur3": "^1.0.3",
"@ipld/dag-cbor": "^8.0.0",
"@ipld/dag-pb": "^3.0.0",
"@multiformats/murmur3": "^2.0.0",
"err-code": "^3.0.1",
"hamt-sharding": "^3.0.0",
"interface-blockstore": "^3.0.0",
"ipfs-unixfs": "^7.0.0",
"it-last": "^1.0.5",
"it-parallel": "^2.0.1",
"it-last": "^2.0.0",
"it-map": "^2.0.0",
"it-parallel": "^3.0.0",
"it-pipe": "^2.0.4",
"it-pushable": "^3.1.0",
"it-map": "^1.0.6",
"multiformats": "^10.0.0",
"p-queue": "^7.3.0",
"multiformats": "^9.4.2",
"uint8arrays": "^3.0.0"
"uint8arrays": "^4.0.2"
},
"devDependencies": {
"@types/sinon": "^10.0.0",
Expand All @@ -175,9 +175,9 @@
"crypto-browserify": "^3.12.0",
"delay": "^5.0.0",
"ipfs-unixfs-importer": "^10.0.0",
"it-all": "^1.0.5",
"it-buffer-stream": "^2.0.0",
"it-first": "^1.0.6",
"it-all": "^2.0.0",
"it-buffer-stream": "^3.0.0",
"it-first": "^2.0.0",
"merge-options": "^3.0.4",
"native-abort-controller": "^1.0.3",
"sinon": "^14.0.0"
Expand Down
29 changes: 16 additions & 13 deletions packages/ipfs-unixfs-importer/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# ipfs-unixfs-importer <!-- omit in toc -->

[![ipfs.io](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io)
[![IRC](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
[![Discord](https://img.shields.io/discord/806902334369824788?style=flat-square)](https://discord.gg/ipfs)
[![ipfs.tech](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](https://ipfs.tech)
[![Discuss](https://img.shields.io/discourse/https/discuss.ipfs.tech/posts.svg?style=flat-square)](https://discuss.ipfs.tech)
[![codecov](https://img.shields.io/codecov/c/github/ipfs/js-ipfs-unixfs.svg?style=flat-square)](https://codecov.io/gh/ipfs/js-ipfs-unixfs)
[![CI](https://img.shields.io/github/workflow/status/ipfs/js-ipfs-unixfs/test%20&%20maybe%20release/master?style=flat-square)](https://github.com/ipfs/js-ipfs-unixfs/actions/workflows/js-test-and-release.yml)

Expand Down Expand Up @@ -167,14 +166,6 @@ Several aspects of the importer are overridable by specifying functions as part
- This function should read `{ cid, path, unixfs, node }` entries from `source` and place them in a directory structure
- It should yield an object with the properties `{ cid, path, unixfs, size }` where `cid` is a `CID`, `path` is a string, `unixfs` is a UnixFS entry and `size` is a `Number`.

[blockstore]: https://github.com/ipfs/js-ipfs-interfaces/tree/master/packages/interface-blockstore#readme

[UnixFS]: https://github.com/ipfs/specs/tree/master/unixfs

[IPLD]: https://github.com/ipld/js-ipld

[CID]: https://github.com/multiformats/js-cid

## Contribute

Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-importer/issues)!
Expand All @@ -192,8 +183,20 @@ Licensed under either of

## Contribute

Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-importer/issues)!
Contributions welcome! Please check out [the issues](https://github.com/ipfs/js-ipfs-unixfs/issues).

This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
Also see our [contributing document](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md) for more information on how we work, and about contributing in general.

Please be aware that all interactions related to this repo are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

[![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)

[blockstore]: https://github.com/ipfs/js-ipfs-interfaces/tree/master/packages/interface-blockstore#readme

[UnixFS]: https://github.com/ipfs/specs/tree/master/unixfs

[IPLD]: https://github.com/ipld/js-ipld

[CID]: https://github.com/multiformats/js-cid
20 changes: 10 additions & 10 deletions packages/ipfs-unixfs-importer/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -151,27 +151,27 @@
"release": "aegir release"
},
"dependencies": {
"@ipld/dag-pb": "^2.0.2",
"@multiformats/murmur3": "^1.0.3",
"bl": "^5.0.0",
"@ipld/dag-pb": "^3.0.0",
"@multiformats/murmur3": "^2.0.0",
"err-code": "^3.0.1",
"hamt-sharding": "^3.0.0",
"interface-blockstore": "^3.0.0",
"ipfs-unixfs": "^7.0.0",
"it-all": "^1.0.5",
"it-batch": "^1.0.8",
"it-first": "^1.0.6",
"it-parallel-batch": "^1.0.9",
"it-all": "^2.0.0",
"it-batch": "^2.0.0",
"it-first": "^2.0.0",
"it-parallel-batch": "^2.0.0",
"merge-options": "^3.0.4",
"multiformats": "^9.4.2",
"multiformats": "^10.0.0",
"rabin-wasm": "^0.1.4",
"uint8arrays": "^3.0.0"
"uint8arraylist": "^2.3.3",
"uint8arrays": "^4.0.2"
},
"devDependencies": {
"aegir": "^37.5.0",
"assert": "^2.0.0",
"blockstore-core": "^2.0.1",
"it-buffer-stream": "^2.0.0",
"it-buffer-stream": "^3.0.0",
"wherearewe": "^2.0.1"
},
"browser": {
Expand Down
21 changes: 10 additions & 11 deletions packages/ipfs-unixfs-importer/src/chunker/fixed-size.js
Original file line number Diff line number Diff line change
@@ -1,32 +1,31 @@
// @ts-ignore
import BufferList from 'bl/BufferList.js'
import { Uint8ArrayList } from 'uint8arraylist'

/**
* @type {import('../types').Chunker}
*/
async function * fixedSizeChunker (source, options) {
let bl = new BufferList()
let list = new Uint8ArrayList()
let currentLength = 0
let emitted = false
const maxChunkSize = options.maxChunkSize

for await (const buffer of source) {
bl.append(buffer)
list.append(buffer)

currentLength += buffer.length

while (currentLength >= maxChunkSize) {
yield bl.slice(0, maxChunkSize)
yield list.slice(0, maxChunkSize)
emitted = true

// throw away consumed bytes
if (maxChunkSize === bl.length) {
bl = new BufferList()
if (maxChunkSize === list.length) {
list = new Uint8ArrayList()
currentLength = 0
} else {
const newBl = new BufferList()
newBl.append(bl.shallowSlice(maxChunkSize))
bl = newBl
const newBl = new Uint8ArrayList()
newBl.append(list.sublist(maxChunkSize))
list = newBl

// update our offset
currentLength -= maxChunkSize
Expand All @@ -36,7 +35,7 @@ async function * fixedSizeChunker (source, options) {

if (!emitted || currentLength) {
// return any remaining bytes or an empty buffer
yield bl.slice(0, currentLength)
yield list.subarray(0, currentLength)
}
}

Expand Down
7 changes: 3 additions & 4 deletions packages/ipfs-unixfs-importer/src/chunker/rabin.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
// @ts-ignore
import BufferList from 'bl/BufferList.js'
import { Uint8ArrayList } from 'uint8arraylist'
// @ts-ignore
import { create } from 'rabin-wasm'
import errcode from 'err-code'
Expand Down Expand Up @@ -65,7 +64,7 @@ export default rabinChunker
*/
async function * rabin (source, options) {
const r = await create(options.bits, options.min, options.max, options.window)
const buffers = new BufferList()
const buffers = new Uint8ArrayList()

for await (const chunk of source) {
buffers.append(chunk)
Expand All @@ -82,6 +81,6 @@ async function * rabin (source, options) {
}

if (buffers.length) {
yield buffers.slice(0)
yield buffers.subarray(0)
}
}
2 changes: 1 addition & 1 deletion packages/ipfs-unixfs-importer/src/types.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import type { UnixFS, Mtime } from 'ipfs-unixfs'
import type { CID, CIDVersion } from 'multiformats/cid'
import type { CID, Version as CIDVersion } from 'multiformats/cid'
import type { MultihashHasher } from 'multiformats/hashes/interface'
import type { BlockCodec } from 'multiformats/codecs/interface'
import type { Blockstore } from 'interface-blockstore'
Expand Down
13 changes: 8 additions & 5 deletions packages/ipfs-unixfs/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# ipfs-unixfs <!-- omit in toc -->

[![ipfs.io](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io)
[![IRC](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
[![Discord](https://img.shields.io/discord/806902334369824788?style=flat-square)](https://discord.gg/ipfs)
[![ipfs.tech](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](https://ipfs.tech)
[![Discuss](https://img.shields.io/discourse/https/discuss.ipfs.tech/posts.svg?style=flat-square)](https://discuss.ipfs.tech)
[![codecov](https://img.shields.io/codecov/c/github/ipfs/js-ipfs-unixfs.svg?style=flat-square)](https://codecov.io/gh/ipfs/js-ipfs-unixfs)
[![CI](https://img.shields.io/github/workflow/status/ipfs/js-ipfs-unixfs/test%20&%20maybe%20release/master?style=flat-square)](https://github.com/ipfs/js-ipfs-unixfs/actions/workflows/js-test-and-release.yml)

Expand Down Expand Up @@ -213,8 +212,12 @@ Licensed under either of

## Contribute

Feel free to join in. All welcome. Open an [issue](https://github.com/ipfs/js-ipfs-unixfs-importer/issues)!
Contributions welcome! Please check out [the issues](https://github.com/ipfs/js-ipfs-unixfs/issues).

This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
Also see our [contributing document](https://github.com/ipfs/community/blob/master/CONTRIBUTING_JS.md) for more information on how we work, and about contributing in general.

Please be aware that all interactions related to this repo are subject to the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

[![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)
2 changes: 1 addition & 1 deletion packages/ipfs-unixfs/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@
"devDependencies": {
"aegir": "^37.5.0",
"protobufjs-cli": "^1.0.0",
"uint8arrays": "^3.0.0"
"uint8arrays": "^4.0.2"
},
"browser": {
"fs": false
Expand Down