Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add noirc_abi_wasm crate for ABI encoding in JS #1945

Merged
merged 49 commits into from
Sep 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
e611498
chore: temp commit
TomAFrench Jul 17, 2023
90be749
chore: update package.json
TomAFrench Jul 18, 2023
901314d
fix: package name and paths for build.sh
kobyhallx Jul 18, 2023
e2775a4
feat: add another package to flake
kobyhallx Jul 18, 2023
38b7569
fix: shell env to 1.66.1
kobyhallx Jul 18, 2023
c468e17
chore: refactor scripts
TomAFrench Jul 18, 2023
8d615e4
chore: add test framework for new abi package
TomAFrench Jul 18, 2023
04ac605
chore: remove unused dependencies
TomAFrench Jul 19, 2023
0bc9944
chore: run abi encoding tests in CI
TomAFrench Jul 19, 2023
cbcca58
chore: fix typo in new workflow
TomAFrench Jul 19, 2023
b1b218f
chore: allow running `yarn install` in a non-root directory
TomAFrench Jul 19, 2023
6f4b098
chore: standardise triggers for workflow
TomAFrench Jul 19, 2023
616530f
chore: fix action yaml
TomAFrench Jul 19, 2023
91f46c2
chore: fix inputs
TomAFrench Jul 19, 2023
6063d64
chore: move cd to same step as yarn install
TomAFrench Jul 19, 2023
d7579c6
feat: add devcontainer config
kobyhallx Jul 19, 2023
4432308
Merge branch 'abi-wasm' of github.com:noir-lang/noir into abi-wasm
kobyhallx Jul 19, 2023
0ccf055
Merge branch 'master' into abi-wasm
TomAFrench Aug 1, 2023
285ee1a
Merge branch 'master' into abi-wasm
TomAFrench Aug 10, 2023
6df8af1
chore: pull across build process from `noir_wasm`
TomAFrench Aug 10, 2023
197c961
chore: test on firefox
TomAFrench Aug 10, 2023
b90b84f
chore: fix typo in README
TomAFrench Aug 10, 2023
075fde8
Merge branch 'master' into abi-wasm
TomAFrench Aug 15, 2023
4182a68
chore: update `noirc_abi_wasm` build scripts
TomAFrench Aug 15, 2023
f261c73
chore: fix merge
TomAFrench Aug 15, 2023
f3b28c0
chore: Cargo.toml tweaks
TomAFrench Aug 15, 2023
75d2137
chore: Fix typo (#2315)
kevaundray Aug 15, 2023
48b20fa
chore: Make `wasm` tests pull from `result` directory (#2319)
TomAFrench Aug 15, 2023
2684b56
chore: mirror `wasm` build scripts
TomAFrench Aug 15, 2023
9ed9fd8
chore: remove `.devcontainer.json`
TomAFrench Aug 15, 2023
311dc99
chore: fix build command
TomAFrench Aug 15, 2023
1ca0ce2
Merge branch 'master' into abi-wasm
TomAFrench Aug 15, 2023
c1d8623
chore: update package name in build script
TomAFrench Aug 15, 2023
d8b8c93
chore: revert change to wasm build script
TomAFrench Aug 15, 2023
e0edbd6
chore: fix package name
TomAFrench Aug 15, 2023
c970b4f
Merge branch 'master' into abi-wasm
TomAFrench Aug 17, 2023
a71e5a7
chore: revert changes to flake.nix
TomAFrench Aug 17, 2023
ec855ff
chore: fix caching of `noirc_abi_wasm`
TomAFrench Aug 17, 2023
1bfab41
chore(build): add wasm-opt
kobyhallx Sep 1, 2023
778af65
wip: getting abiEncoder to compile
kobyhallx Sep 4, 2023
41720c6
Merge branch 'master' into abi-wasm
kobyhallx Sep 5, 2023
0f87c5c
chore: adjust package paths
kobyhallx Sep 5, 2023
dc8324a
Merge branch 'master' into abi-wasm
kobyhallx Sep 5, 2023
bc21e14
fix: cache issue
kobyhallx Sep 5, 2023
43cc363
chore: bump nix pkgs version
kobyhallx Sep 5, 2023
b005033
Update crates/noirc_abi_wasm/test/browser/abi_encode.test.ts
kobyhallx Sep 5, 2023
05e86ef
chore: resolving nits
kobyhallx Sep 5, 2023
3eaea24
Merge branch 'abi-wasm' of github.com:noir-lang/noir into abi-wasm
kobyhallx Sep 5, 2023
1182be1
chore: retain artifact for 10d instead of 3d
kobyhallx Sep 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/actions/setup/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Setup

inputs:
working-directory:
default: ./
required: false

runs:
using: composite
steps:
- name: Install node
uses: actions/setup-node@v3
with:
node-version: 18.15
- name: Cache
uses: actions/cache@v3
id: cache
with:
path: "**/node_modules"
key: yarn-v1-${{ hashFiles('**/yarn.lock') }}
- name: Install
run: |
cd ${{ inputs.working-directory }}
yarn --immutable
shell: bash
if: steps.cache.outputs.cache-hit != 'true'
122 changes: 122 additions & 0 deletions .github/workflows/abi_wasm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
name: ABI Wasm test

on:
pull_request:
merge_group:

# This will cancel previous runs when a branch or PR is updated
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.ref || github.run_id }}
cancel-in-progress: true

jobs:
noirc-abi-wasm-build:
runs-on: ubuntu-latest
env:
CACHED_PATH: /tmp/nix-cache

steps:
- name: Checkout sources
uses: actions/checkout@v3

- uses: cachix/install-nix-action@v20
with:
nix_path: nixpkgs=channel:nixos-23.05
github_access_token: ${{ secrets.GITHUB_TOKEN }}

- name: Restore nix store cache
uses: actions/cache/restore@v3
id: cache
with:
path: ${{ env.CACHED_PATH }}
key: ${{ runner.os }}-flake-abi-wasm-${{ hashFiles('*.lock') }}

# Based on https://github.com/marigold-dev/deku/blob/b5016f0cf4bf6ac48db9111b70dd7fb49b969dfd/.github/workflows/build.yml#L26
- name: Copy cache into nix store
if: steps.cache.outputs.cache-hit == 'true'
# We don't check the signature because we're the one that created the cache
run: |
for narinfo in ${{ env.CACHED_PATH }}/*.narinfo; do
path=$(head -n 1 "$narinfo" | awk '{print $2}')
nix copy --no-check-sigs --from "file://${{ env.CACHED_PATH }}" "$path"
done

- name: Build noirc_abi_wasm
run: |
nix build -L .#noirc_abi_wasm

- name: Export cache from nix store
if: ${{ steps.cache.outputs.cache-hit != 'true' && github.event_name != 'merge_group' }}
run: |
nix copy --to "file://${{ env.CACHED_PATH }}?compression=zstd&parallel-compression=true" .#noirc-abi-wasm-cargo-artifacts

- uses: actions/cache/save@v3
# Don't create cache entries for the merge queue.
if: ${{ steps.cache.outputs.cache-hit != 'true' && github.event_name != 'merge_group' }}
with:
path: ${{ env.CACHED_PATH }}
key: ${{ steps.cache.outputs.cache-primary-key }}

- name: Dereference symlink
run: echo "UPLOAD_PATH=$(readlink -f result)" >> $GITHUB_ENV

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: noirc_abi_wasm
path: ${{ env.UPLOAD_PATH }}
retention-days: 10

noirc-abi-wasm-test-node:
needs: [noirc-abi-wasm-build]
name: Node.js Tests
runs-on: ubuntu-latest

steps:
- name: Checkout sources
uses: actions/checkout@v3

- name: Download artifact
uses: actions/download-artifact@v3
with:
name: noirc_abi_wasm
path: ./result

- name: Set up test environment
uses: ./.github/actions/setup
with:
working-directory: ./crates/noirc_abi_wasm

- name: Run node tests
working-directory: ./crates/noirc_abi_wasm
run: yarn test

noirc-abi-wasm-test-browser:
needs: [noirc-abi-wasm-build]
name: Browser Tests
runs-on: ubuntu-latest

steps:
- name: Checkout sources
uses: actions/checkout@v3

- name: Download artifact
uses: actions/download-artifact@v3
with:
name: noirc_abi_wasm
path: ./result

- name: Set up test environment
uses: ./.github/actions/setup
with:
working-directory: ./crates/noirc_abi_wasm

- name: Install playwright deps
working-directory: ./crates/noirc_abi_wasm
run: |
npx playwright install
npx playwright install-deps

- name: Run browser tests
working-directory: ./crates/noirc_abi_wasm
run: yarn test:browser
47 changes: 47 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ members = [
"crates/fm",
"crates/arena",
"crates/noirc_abi",
"crates/noirc_abi_wasm",
"crates/iter-extended",
"crates/wasm",
]
Expand Down
2 changes: 2 additions & 0 deletions crates/noirc_abi_wasm/.eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules
pkg
19 changes: 19 additions & 0 deletions crates/noirc_abi_wasm/.eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
module.exports = {
root: true,
parser: "@typescript-eslint/parser",
plugins: ["@typescript-eslint", "prettier"],
extends: ["eslint:recommended", "plugin:@typescript-eslint/recommended"],
rules: {
"comma-spacing": ["error", { before: false, after: true }],
"no-unused-vars": "off",
"@typescript-eslint/no-unused-vars": [
"warn", // or "error"
{
argsIgnorePattern: "^_",
varsIgnorePattern: "^_",
caughtErrorsIgnorePattern: "^_",
},
],
"prettier/prettier": "error",
},
};
8 changes: 8 additions & 0 deletions crates/noirc_abi_wasm/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Yarn
.pnp.*
.yarn/*
!.yarn/patches
!.yarn/plugins
!.yarn/releases
!.yarn/sdks
!.yarn/versions
5 changes: 5 additions & 0 deletions crates/noirc_abi_wasm/.mocharc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"extension": ["ts"],
"spec": "test/node/**/*.test.ts",
"require": "ts-node/register"
}
873 changes: 873 additions & 0 deletions crates/noirc_abi_wasm/.yarn/releases/yarn-3.5.1.cjs

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions crates/noirc_abi_wasm/.yarnrc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
yarnPath: .yarn/releases/yarn-3.5.1.cjs

nodeLinker: node-modules
Empty file.
33 changes: 33 additions & 0 deletions crates/noirc_abi_wasm/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
[package]
name = "noirc_abi_wasm"
version.workspace = true
authors.workspace = true
edition.workspace = true

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html


[lib]
crate-type = ["cdylib"]

[dependencies]
acvm.workspace = true
noirc_abi.workspace = true
iter-extended.workspace = true
wasm-bindgen.workspace = true
serde.workspace = true

console_error_panic_hook = "0.1.7"
gloo-utils = { version = "0.1", features = ["serde"] }

js-sys = "0.3.62"

# This is an unused dependency, we are adding it
# so that we can enable the js feature in getrandom.
getrandom = { version = "*", features = ["js"] }

[build-dependencies]
build-data = "0.1.3"

[dev-dependencies]
wasm-bindgen-test = "0.3.36"
17 changes: 17 additions & 0 deletions crates/noirc_abi_wasm/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Noir Lang ABI JavaScript Package

This JavaScript package enables users to ABI encode inputs to a Noir program, i.e. generating an initial witness.

## Building from source

Outside of the [noir repo](https://github.com/noir-lang/noir), this package can be built using the command below:

```bash
nix build -L github:noir-lang/noir/master#abi_wasm
```

If you are within the noir repo and would like to build local changes, you can use:

```bash
nix build -L #abi_wasm
```
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
14 changes: 14 additions & 0 deletions crates/noirc_abi_wasm/build.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
const GIT_COMMIT: &&str = &"GIT_COMMIT";

fn main() {
// Only use build_data if the environment variable isn't set
// The environment variable is always set when working via Nix
if std::env::var(GIT_COMMIT).is_err() {
build_data::set_GIT_COMMIT();
build_data::set_GIT_DIRTY();
build_data::no_debug_rebuilds();
}

build_data::set_SOURCE_TIMESTAMP();
build_data::no_debug_rebuilds();
}
48 changes: 48 additions & 0 deletions crates/noirc_abi_wasm/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
#!/usr/bin/env bash

function require_command {
if ! command -v "$1" >/dev/null 2>&1; then
echo "Error: $1 is required but not installed." >&2
exit 1
fi
}
function check_installed {
if ! command -v "$1" >/dev/null 2>&1; then
echo "$1 is not installed. Please install it." >&2
return 1
fi
return 0
}
function run_or_fail {
"$@"
local status=$?
if [ $status -ne 0 ]; then
echo "Command '$*' failed with exit code $status" >&2
exit $status
fi
}

require_command toml2json
require_command jq
require_command cargo
require_command wasm-bindgen
require_command wasm-opt

self_path=$(dirname "$(readlink -f "$0")")
export pname=$(toml2json < ${self_path}/Cargo.toml | jq -r .package.name)
export CARGO_TARGET_DIR=$self_path/target

rm -rf $self_path/outputs >/dev/null 2>&1
rm -rf $self_path/result >/dev/null 2>&1

if [ -v out ]; then
echo "Will install package to $out (defined outside installPhase.sh script)"
else
out="$self_path/outputs/out"
echo "Will install package to $out"
fi

run_or_fail ${self_path}/buildPhaseCargoCommand.sh release
run_or_fail ${self_path}/installPhase.sh

ln -s $out $self_path/result
Loading