Refactor test gen

Cargo.toml

@@ -1,5 +1,8 @@
 [workspace]
+resolver = "2"
+
 members = [
     "ssz-rs",
     "ssz-rs-derive",
+    "ssz-rs-test-gen"
 ]

README.md

@@ -16,22 +16,26 @@ If you need a battle-tested implementation (e.g. for consensus-critical work), r
 
 # Features
 
-To conform to the SSZ spec, a given Rust type should implement the `SimpleSerialize` trait. Types implementing this trait then obtain:
+To conform to the `SSZ` spec, a given Rust type should implement the [`SimpleSerialize` trait](https://docs.rs/ssz_rs/latest/ssz_rs/trait.SimpleSerialize.html). Types implementing this trait then obtain:
 
 ## Encoding / decoding
 
-`ssz_rs` aims to add as little ceremony over the built-in Rust types as possible. The `ssz_rs_derive` crate provides macros to derive the encoding and decoding routines for SSZ containers and unions (represented as Rust `struct`s and `enum`s, respectively).
+`ssz_rs` aims to add as little ceremony over the built-in Rust types as possible.
+The `ssz_rs_derive` crate provides macros to derive the encoding and decoding routines for SSZ containers and unions (represented as Rust `struct`s and `enum`s, respectively).
 See the `ssz_rs/examples` for example usage.
 
 ## Merkleization
 
 This library provides the hash tree root computation for types implementing `SimpleSerialize`.
 
+* *NOTE*: more sophisticated caching strategies are possible, users may run into memory issues with the current implementation.
+
 ## Multiproofs
 
-* *NOTE*: under construction
+This library provides the ability to reason about [generalized indices](https://github.com/ethereum/consensus-specs/blob/dev/ssz/merkle-proofs.md#generalized-merkle-tree-index) for a given `SSZ` definition,
+along with the ability to generate and verify proofs of data at those indices.
 
-This library provides tools for generating and verifying multiproofs of SSZ data.
+* *NOTE*: still under construction
 
 ## `no-std` feature
 
@@ -40,11 +44,10 @@ This library is `no-std` compatible. To build without the standard library, disa
 For example, in `Cargo.toml`:
 
 ```toml
-ssz-rs = { version = "...", default-features = false }
+ssz_rs = { version = "...", default-features = false }
 ```
 
 # Testing
 
-This repo includes a copy of the [`ssz_generic` consensus spec tests](https://github.com/ethereum/consensus-spec-tests) as integration tests for the `ssz_rs` crate.
-The tests are generated from a local clone of the spec tests repo and the generator script under `ssz_rs/scripts`.
-Refer to the README there if you need to update/change these tests.
+This repo includes a copy of the [`ssz_generic` consensus spec tests](https://github.com/ethereum/consensus-spec-tests) as integration tests for the `ssz_rs` crate, along with hand-written unit tests.
+The integration tests are generated via a utility under `ssz-rs-test-gen` package. See the README there for further details.

ssz-rs-test-gen/Cargo.toml

@@ -0,0 +1,15 @@
+[package]
+name = "ssz-rs-test-gen"
+version = "0.1.0"
+edition = "2021"
+license = "MIT OR Apache-2.0"
+readme = "./README.md"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+serde = "1.0"
+serde_yaml = "0.9"
+num-bigint = "0.4.3"
+hex = "0.4.3"
+convert_case = "0.6.0"

ssz-rs-test-gen/README.md

@@ -0,0 +1,11 @@
+# ssz-rs-test-gen
+
+Utility for generating the integration tests for `ssz_rs`.
+
+## How to use
+
+```bash
+just clean
+just download-integration-tests
+just generate-all
+```

ssz-rs-test-gen/justfile

@@ -0,0 +1,33 @@
+default-fmt := "true"
+
+generate-all:
+    #!/usr/bin/env sh
+    for TYP in basic_vector bitlist bitvector boolean containers uints; do
+        just generate $TYP false
+    done;
+    just fmt
+
+generate TYPE FMT=default-fmt:
+    cargo run -p ssz-rs-test-gen {{TYPE}}
+    if {{FMT}} == true; then just fmt; fi
+
+fmt:
+    cargo +nightly fmt --all
+
+generate-clean:
+    rm -rf ../ssz-rs/tests/data
+
+download-integration-tests: integration-tests-clean
+    #!/usr/bin/env sh
+    TESTS_TAG=$(cat spec-test-version)
+    REPO_NAME=consensus-spec-tests
+    CONFIG="general"
+    mkdir ${REPO_NAME}
+    wget https://github.com/ethereum/${REPO_NAME}/releases/download/${TESTS_TAG}/${CONFIG}.tar.gz
+    tar -xzf ${CONFIG}.tar.gz -C ${REPO_NAME}
+    rm -f *tar.gz
+
+integration-tests-clean:
+    rm -rf consensus-spec-tests
+
+clean: generate-clean integration-tests-clean

ssz-rs-test-gen/spec-test-version

@@ -0,0 +1 @@
+v1.3.0