Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

const generics #116

Closed
wants to merge 22 commits into from
Closed
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ derive = ["multihash-derive"]
arb = ["quickcheck", "rand"]
secure-hashes = ["blake2b", "blake2s", "blake3", "sha2", "sha3"]
scale-codec = ["parity-scale-codec"]
serde-codec = ["serde", "generic-array/serde"]
serde-codec = ["serde", "serde-big-array"]

blake2b = ["blake2b_simd"]
blake2s = ["blake2s_simd"]
Expand All @@ -32,11 +32,11 @@ sha3 = ["digest", "sha-3"]
strobe = ["strobe-rs"]

[dependencies]
generic-array = "0.14.4"
parity-scale-codec = { version = "1.3.5", optional = true, default-features = false, features = ["derive"] }
quickcheck = { version = "0.9.2", optional = true }
rand = { version = "0.7.3", optional = true }
serde = { version = "1.0.116", optional = true, default-features = false, features = ["derive"] }
serde-big-array = { version = "0.3.2", optional = true, features = ["const-generics"] }
multihash-derive = { version = "^0.7.1", path = "derive", default-features = false, optional = true }
unsigned-varint = "0.7.0"

Expand Down
12 changes: 6 additions & 6 deletions derive/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@
//!
//! If you set `#mh(alloc_size = …)` to a too low value, you will get compiler errors. Please note
//! the the sizes are checked only on a syntactic level and *not* on the type level. This means
//! that digest need to have a size generic, which is a valid `typenum`, for example `U32` or
//! `generic_array::typenum::U64`.
//! that digest need to have a size const generic, which is a valid `usize`, for example `32` or
//! `64`.
//!
//! You can disable those compiler errors with setting the `no_alloc_size_errors` attribute. This
//! can be useful if you e.g. have specified type aliases for your hash digests and you are sure
Expand All @@ -19,14 +19,14 @@
//!
//! ```
//! use multihash::derive::Multihash;
//! use multihash::{MultihashDigest, U32, U64};
//! use multihash::MultihashDigest;
//!
//! #[derive(Clone, Copy, Debug, Eq, Multihash, PartialEq)]
//! #[mh(alloc_size = U64)]
//! #[mh(alloc_size = 64)]
//! pub enum Code {
//! #[mh(code = 0x01, hasher = multihash::Sha2_256, digest = multihash::Sha2Digest<U32>)]
//! #[mh(code = 0x01, hasher = multihash::Sha2_256, digest = multihash::Sha2Digest<32>)]
//! Foo,
//! #[mh(code = 0x02, hasher = multihash::Sha2_512, digest = multihash::Sha2Digest<U64>)]
//! #[mh(code = 0x02, hasher = multihash::Sha2_512, digest = multihash::Sha2Digest<64>)]
//! Bar,
//! }
//!
Expand Down
153 changes: 64 additions & 89 deletions derive/src/multihash.rs

Large diffs are not rendered by default.

15 changes: 6 additions & 9 deletions examples/custom_table.rs
Original file line number Diff line number Diff line change
@@ -1,19 +1,16 @@
use std::convert::TryFrom;

use multihash::derive::Multihash;
use multihash::typenum::{U20, U25, U64};
use multihash::{
Digest, Error, Hasher, MultihashDigest, MultihashGeneric, Sha2Digest, Sha2_256, Size,
StatefulHasher,
Digest, Error, Hasher, MultihashDigest, MultihashGeneric, Sha2Digest, Sha2_256, StatefulHasher,
};

// You can implement a custom hasher. This is a SHA2 256-bit hasher that returns a hash that is
// truncated to 160 bits.
#[derive(Default, Debug)]
pub struct Sha2_256Truncated20(Sha2_256);
impl StatefulHasher for Sha2_256Truncated20 {
type Size = U20;
type Digest = Sha2Digest<Self::Size>;
impl StatefulHasher<20> for Sha2_256Truncated20 {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should still be an associated constant

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's Self::SIZE?

Copy link
Contributor Author

@mriise mriise Jun 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self::SIZE replaced size() since while it could be a const fn it made more sense (in my mind) to make it a public associated constant.

It would make sense to move back to const fn size() in the case of having SIZE be an associated constant without being repetitive.

Copy link
Contributor Author

@mriise mriise Jun 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems using an assosicated const as a paramater is still part of the incomplete const_generics feature in nightly, though with enough feature flags we can make it look something like this

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=cfcf3ec66baa902500cae4dbcefd7dff

Copy link
Member

@vmx vmx Jun 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not being able to have it as associate constant in current stable Rust sucks :-/

I guess we leave it like that then, until it;s possible in stable Rust? What do others think?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we in a hurry to merge this? it's a usability regression, and comparing it to the benefits, maybe we should hold off until more of const-generics is stabilized.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not in a hurry. Though I fear that it will take a long time until it's usable on stable.

@mriise What do you think about perhaps fixing everything up a single commit (once we are happy with the state of it) and we just keep it open? Then people who want const generic support can easily cherry-pick it themselves and it should also be easier to rebase it if we make other changes (e.g. fixing the blake3 hashing). This way all your work is not lost, still usable and hopefully find it's way in some day.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although it is disappointing, I would have to agree. I'll do the final bits of cleanup and request another review.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, is this really a regression? Or could we use this to trim down our digest list a bit (e.g., for all the Blake2s variants).

type Digest = Sha2Digest<{ Self::SIZE }>;
fn update(&mut self, input: &[u8]) {
self.0.update(input)
}
Expand All @@ -28,13 +25,13 @@ impl StatefulHasher for Sha2_256Truncated20 {
}

#[derive(Clone, Copy, Debug, Eq, Multihash, PartialEq)]
#[mh(alloc_size = U64)]
#[mh(alloc_size = 64)]
pub enum Code {
/// Example for using a custom hasher which returns truncated hashes
#[mh(code = 0x12, hasher = Sha2_256Truncated20, digest = multihash::Sha2Digest<U20>)]
#[mh(code = 0x12, hasher = Sha2_256Truncated20, digest = multihash::Sha2Digest<20>)]
Sha2_256Truncated20,
/// Example for using a hasher with a bit size that is not exported by default
#[mh(code = 0xb219, hasher = multihash::Blake2bHasher::<U25>, digest = multihash::Blake2bDigest<U25>)]
#[mh(code = 0xb219, hasher = multihash::Blake2bHasher::<25>, digest = multihash::Blake2bDigest<25>)]
Blake2b200,
}

Expand Down
4 changes: 2 additions & 2 deletions src/arb.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ use rand::{
Rng,
};

use crate::{MultihashGeneric, U64};
use crate::MultihashGeneric;

/// Generates a random valid multihash.
impl Arbitrary for MultihashGeneric<U64> {
impl Arbitrary for MultihashGeneric<64> {
fn arbitrary<G: Gen>(g: &mut G) -> Self {
// In real world lower multihash codes are more likely to happen, hence distribute them
// with bias towards smaller values.
Expand Down
56 changes: 16 additions & 40 deletions src/hasher.rs
Original file line number Diff line number Diff line change
@@ -1,25 +1,12 @@
use crate::error::Error;
use core::fmt::Debug;
use generic_array::typenum::marker_traits::Unsigned;
use generic_array::{ArrayLength, GenericArray};

/// Size marker trait.
pub trait Size:
ArrayLength<u8> + Debug + Default + Eq + core::hash::Hash + Send + Sync + 'static
{
}

impl<T: ArrayLength<u8> + Debug + Default + Eq + core::hash::Hash + Send + Sync + 'static> Size
for T
{
}

/// Stack allocated digest trait.
pub trait Digest<S: Size>:
pub trait Digest<const S: usize>:
AsRef<[u8]>
+ AsMut<[u8]>
+ From<GenericArray<u8, S>>
+ Into<GenericArray<u8, S>>
+ From<[u8; S]>
+ Into<[u8; S]>
+ Clone
+ core::hash::Hash
+ Debug
Expand All @@ -29,17 +16,15 @@ pub trait Digest<S: Size>:
+ Sync
+ 'static
{
/// Size of the digest.
fn size(&self) -> u8 {
S::to_u8()
}
/// Size of the digest. Maximum for Some of the Blake family is 2^64-1 bytes
const SIZE: usize = S;

/// Wraps the digest bytes.
fn wrap(digest: &[u8]) -> Result<Self, Error> {
mriise marked this conversation as resolved.
Show resolved Hide resolved
if digest.len() != S::to_usize() {
if digest.len() != S {
return Err(Error::InvalidSize(digest.len() as _));
}
let mut array = GenericArray::default();
let mut array = [0; S];
let len = digest.len().min(array.len());
array[..len].copy_from_slice(&digest[..len]);
Ok(array.into())
Expand All @@ -56,22 +41,19 @@ pub trait Digest<S: Size>:
use unsigned_varint::io::read_u64;

let size = read_u64(&mut r)?;
if size > S::to_u64() || size > u8::max_value() as u64 {
if size > S as u64 || size > u8::max_value() as u64 {
return Err(Error::InvalidSize(size));
}
let mut digest = GenericArray::default();
let mut digest = [0; S];
r.read_exact(&mut digest[..size as usize])?;
Ok(Self::from(digest))
}
}

/// Trait implemented by a hash function implementation.
pub trait StatefulHasher: Default + Send + Sync {
/// The maximum Digest size for that hasher (it is stack allocated).
type Size: Size;

pub trait StatefulHasher<const SIZE: usize>: Default + Send + Sync {
/// The Digest type to distinguish the output of different `Hasher` implementations.
type Digest: Digest<Self::Size>;
type Digest: Digest<SIZE>;

/// Consume input and update internal state.
fn update(&mut self, input: &[u8]);
Expand Down Expand Up @@ -106,26 +88,20 @@ pub trait StatefulHasher: Default + Send + Sync {
/// [Multihashes]: https://github.com/multiformats/multihash
/// [associated type]: https://doc.rust-lang.org/book/ch19-03-advanced-traits.html#specifying-placeholder-types-in-trait-definitions-with-associated-types
/// [`MultihashDigest`]: crate::MultihashDigest
pub trait Hasher: Default + Send + Sync {
/// The maximum Digest size for that hasher (it is stack allocated).
type Size: Size;

pub trait Hasher<const SIZE: usize>: Default + Send + Sync {
/// The Digest type to distinguish the output of different `Hasher` implementations.
type Digest: Digest<Self::Size>;
type Digest: Digest<SIZE>;

/// Returns the allocated size of the digest.
fn size() -> u8 {
Self::Size::to_u8()
}
///the allocated size of the digest.
const SIZE: usize = SIZE;

/// Hashes the given `input` data and returns its hash digest.
fn digest(input: &[u8]) -> Self::Digest
where
Self: Sized;
}

impl<T: StatefulHasher> Hasher for T {
type Size = T::Size;
impl<T: StatefulHasher<SIZE>, const SIZE: usize> Hasher<SIZE> for T {
type Digest = T::Digest;

fn digest(input: &[u8]) -> Self::Digest {
Expand Down
Loading