-
Notifications
You must be signed in to change notification settings - Fork 604
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace fs2.hash with fs2.hashing.Hashing[F] #3454
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
armanbilge
reviewed
Jul 4, 2024
armanbilge
reviewed
Jul 4, 2024
armanbilge
reviewed
Jul 4, 2024
mpilquist
commented
Jul 5, 2024
.make(acquire)(ctx => F.delay(EVP_MD_CTX_free(ctx))) | ||
.evalTap { ctx => | ||
F.delay { | ||
val `type` = EVP_get_digestbyname(toCString(algorithm)(zone)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Initially I thought to free this string right after the call to EVP_get_digestbyname
but the pointer appears to be stored on result struct
mpilquist
changed the title
Initial draft of new hashing package
Replace fs2.hash with fs2.hashing.Hashing[F]
Jul 6, 2024
…endency on MessageDigest)
armanbilge
reviewed
Jul 8, 2024
@armanbilge Thanks, addressed comments |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
fs2.hash
object provides the ability to compute the hash of a stream of bytes (i.e.Stream[F, Byte]
). The hash computation is modeled as aPipe[F, Byte, Byte]
, where pulling on the output stream results in all the source bytes getting hashed and then a final chunk being emitted that contains the hash of the seen bytes.This API is not expressive enough for common use cases though. Consider the case where a
Stream[F, Byte]
needs to be written to a file (or uploaded to an S3 bucket, or sent to a socket, etc) and the hash also needs to be stored. The current hashing API makes this awkward -- either requiring heavy duty machinery likebroadcastThrough
or requiring the stream to be processed twice.This PR deprecates the
fs2.hash
object and replaces it with the newfs2.hashing
package. The entry point to the new package is theHashing[F]
capability trait, allowing the creation ofHash[F]
objects as well as providing various convenience methods.The hashing package contains:
HashAlgorithm
enumeration -- e.g.,HashAlgorithm.SHA256
andHashAlgorithm.Named("MD2")
Hash[F]
mutable object -- allows incremental computation of hashes with a specific algorithm.Hash
defines the following operations:addChunk(c: Chunk[Byte]): F[Unit]
andcomputeAndReset: F[Chunk[Byte]]
update: Pipe[F, Byte, Byte]
- updates the hash with the chunks pulled through the pipeobserve(source: Stream[F, Byte], sink: Pipe[F, Byte, Nothing]): Stream[F, Byte]
- returns a stream that outputs the hash of the source bytes after they've been consumed by the supplied sinkhash: Pipe[F, Byte, Byte]
- outputs the hash of the source bytesverify(expected: Chunk[Byte])(implicit F: RaiseThrowable[F]): Pipe[F, Byte, Byte]
- pipe that outputs the source bytes but raises aHashVerificationException
when the source terminates if the hash of seen bytes doesn't match the expected hashHashing[F]
capability trait -- allows creation ofHash[F]
objectsThe
Hashing[F]
trait returnsHash[F]
objects as resources (i.e.Resource[F, Hash[F]]
) because (on some platforms) they have to be released after computations are complete.With this new API, writing the contents of a source to a file and then subsequently writing a hash to a separate file, while processing the source just once, can be accomplished like so:
The
Hashing
object also contains utility functions for hashing a pure stream and a chunk.