Update to token-types 4.0.1 & strtok3 6.2.2 #466
Conversation
|
@jimmywarting, can you please review? |
| throw new TypeError(`Expected the \`input\` argument to be of type \`Uint8Array\` or \`Buffer\` or \`ArrayBuffer\`, got \`${typeof input}\``); | ||
| } | ||
|
|
||
| const buffer = input instanceof Buffer ? input : Buffer.from(input); | ||
| const buffer = input instanceof Uint8Array ? input : new Uint8Array(input); |
There was a problem hiding this comment.
Had to take a extra look at this, but then i realized that you guard yourself with type checking above.
Buffer.from(input) dose a little bit more magic than new Uint8array(input) like converting strings.
if this is something you also want to support, then a easy way to convert string to uint8 is via new TextEncoder().encode(str)
I just released a package that converts literally anything to Uint8Array with 0 copies if it's of any interest to you... it's called to-uint8array
The Buffer.isBuffer(input) would never be called cuz it would be covered by instanceof uint8 check
There was a problem hiding this comment.
like converting strings.
We wouldn't want that at least. The API is documented to accept a buffer.
There was a problem hiding this comment.
So we allow kind of 3 different input's: Buffer, Uint8Array, ArrayBuffer.
I don't think we need to explicitly add a Buffer here, as Uint8Array is sufficient.
As @sindresorhus indicated, string was not something we had in mind as an input.
There was a problem hiding this comment.
I didn't expect it to work with strings either.
Just wanted to recommend my package cuz it can convert any ArrayBufferView (including Buffer + DataView) & ArrayBuffer and turn them into uint8arrays without copying the memory
Quite handy for any typed array
The updated dependencies try to abstract Buffer dependency by using Uint8Array where possible. Remove Buffer as type, as Buffer is an Uint8Array as well. Add @tokenizer/token to development dependencies. This was not required in the past as @tokenizer/token was incorrectly included indirectly as a normal dependency.
Borewit
force-pushed
the
update-strtok-6.2.2-token-types-4.0.1
branch
from
a4ef125
to
58cb0f3
Compare
Changes:
Uint8Arraywhere possible.Bufferas type, asBufferis a sublcass ofUint8Array.Dependent PR: Borewit/music-metadata#852, to align dependencies token-types & strtok3
Dependency on
Bufferis still there, but in some dependent module theBufferdependency has been entirely abstracted toUint8Array. Let's say I try to get rid of Buffer dependencies if not really required.By doing this, I hope on the long run I hope modules will be better suitable for multipurpose (Node.js & browser) usage.
Fixes: