Skip to content

Commit 824fa98

Browse files
gianbelinchepbezamatias-gonzArtemka374perekopskiy
authored
Hyperchain da removal (#259)
* fix(tee-prover): passthrough env vars to the SGX enclave (matter-labs#2824) ## What ❔ Passthrough env vars to the SGX enclave. Relevant logs showcasing the issue: https://grafana.matterlabs.dev/goto/1iFHMIeIg?orgId=1 ## Why ❔ To fix the bug. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. * feat(zk_toolbox): Add setup keys step to prover init (matter-labs#2811) ## What ❔ Add setup keys step to prover init * feat(zk_toolbox): Allow running docker images for provers (matter-labs#2800) ## What ❔ Add `--docker` flag for `zk_inception prover run` which will allow prover components to run from docker images. ## Why ❔ To decrease setup time/improve UX ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. * chore: Add README for verified sources fetcher (matter-labs#2829) ## What ❔ Adds README for verified sources fetcher ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. * chore(ci): Limit tokio/rayon pools for zk_toolbox CI (matter-labs#2828) ## What ❔ Limits rayon threadpool size to 2 and tokio threadpool size to 4 in zk_toolbox CI. I have checked locally, and with this configuration time to run integration tests is pretty close to the default configuration. ## Why ❔ By default, both tokio and rayon will try to use all the CPUs. When we run multiple Rust binaries at the same time (3 servers and 3 ENs in our case), it causes a lot of conflict for resources, regardless of the number of CPUs. * feat: attester committees data extractor (BFT-434) (matter-labs#2684) Extraction of the attester committee from consensus registry state. If consensus registry address is not specified, we fall back to attester committee from genesis. This pr does NOT enable the dynamic attestation, as the registry address needs to be added to the main node config first. --------- Co-authored-by: Moshe Shababo <17073733+moshababo@users.noreply.github.com> Co-authored-by: Igor Aleksanov <popzxc@yandex.ru> * test: Improve revert integration test (matter-labs#2822) ## What ❔ - Removes a data race from the revert integration test (a single node one); a batch gets executed after it is verified that not all batches are executed. - Removes copy-paste between single-node and EN tests. - Structures revert tests into smaller steps. ## Why ❔ Makes the test less flaky and easier to maintain. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. * test: Fix "missing revert data" error; fix / debug integration tests (matter-labs#2804) ## What ❔ - Fixes the "missing revert data" error by updating the used reth Docker image. The error is probably caused by [this issue](paradigmxyz/reth#7381) fixed in the new reth versions. - Removes "web3 API compatibility tests › Should check API returns error when there are too many logs in eth_getLogs" test as fundamentally flaky and able to poison other tests. - Adds logging for upgrade test to investigate L1 "nonce too low" errors. ## Why ❔ Flaky CI bad. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. * refactor(api): Extract oneshot VM executor to executor crate (matter-labs#2806) ## What ❔ Extracts oneshot VM executor to the executor crate. ## Why ❔ To make executor logic more reusable and maintainable. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. * fix(en): Fix connection starvation during snapshot recovery (matter-labs#2836) ## What ❔ - Fixes DB connection starvation during snapshot recovery. Caused by the insufficient number of connections in the DB pool provided to recovery logic. - Additionally, fixes max concurrency of recovery not being set. ## Why ❔ Connection starvation errors degrade UX. ## Checklist - [x] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [x] Tests for the changes have been added / updated. - [x] Documentation comments have been added / updated. - [x] Code has been formatted via `zk fmt` and `zk lint`. * fix: handling of HTTP 403 thrown by proxyd (matter-labs#2835) When a method is missing: proxyd returns HTTP 403: methodnotfound while api server returns HTTP 200: methodnotfound we need to handle both. * chore: log the whole error message when the task fails (matter-labs#2834) "{err}" is not useful, because it truncates the stack trace of anyhow errors. * feat: allow specifying what tests to run with zks (matter-labs#2841) `zks t i` now accepts a `-t` flag that can can be used to specify a pattern. Only matching tests are run. * chore: clean up dependencies (matter-labs#2839) ## What ❔ Resolves matter-labs#2783 ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. * feat(utils): Rework locate_workspace, introduce Workspace type (matter-labs#2830) ## What - Removes `locate_workspace` and `workspace_dir_or_current_dir` methods. - Instead, introduces `Workspace` type that is aware of different Cargo workspaces in the codebase. ## Why The approach with a single `locate_workspace` doesn't work well for our codebase, since we have multiple workspaces. It resulted in some very implicit and convoluted code (see the removed `get_base_dir` in prover workspace). New approach handles all 3 workspaces _plus_ the lack of a workspace. * feat(zk_toolbox): `zk_supervisor prover` subcommand (matter-labs#2820) ## What ❔ Add prover subcommand for `zk_supervisor`. Add the following subcommand: * `zk_supervisor prover info` - Prints information about current prover setup. * `zk_supervisor prover insert-version` - Insert new protocol version in prover database(integration with `prover_cli`). * `zk_supervisor prover insert-batch` - Insert new batch in prover database(integration with `prover_cli`). Add automatic creation of `prover/artifacts/witness_inputs` dirs if the storage is file backed on init. ## Why ❔ To improve UX of working with provers. ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. * fix(zk-toolbox): Make token multiplier optional (matter-labs#2843) ## What ❔ <!-- What are the changes this PR brings about? --> <!-- Example: This PR adds a PR template to the repo. --> <!-- (For bigger PRs adding more context is appreciated) --> ## Why ❔ <!-- Why are these changes done? What goal do they contribute to? What are the principles behind them? --> <!-- Example: PR templates ensure PR reviewers, observers, and future iterators are in context about the evolution of repos. --> ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. Signed-off-by: Danil <deniallugo@gmail.com> * feat: Smaller zk_environment image (matter-labs#1920) ## What ❔ * Reduced uncompressed size from 7.5GB to 4.81 GB. * Changes to make zk_environment image smaller * compile solc with release mode * remove unnecessary components from google cli * build things first, and then only copy the artifacts. ## Why ❔ * This image is used on ALL of our CI builds - so reducing its size will speed up the CIs. * fix(zk_toolbox): secrets path, artifacts path (matter-labs#2850) ## What ❔ Fix getting chain config in zk_supervisor prover Fix artifacts path when initializing. Setup data path for provers ## Why ❔ <!-- Why are these changes done? What goal do they contribute to? What are the principles behind them? --> <!-- Example: PR templates ensure PR reviewers, observers, and future iterators are in context about the evolution of repos. --> ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. * ci: fix using cargo nextest (matter-labs#2855) ## What ❔ `cargo install --list` doesn't work properly in CI because we copy only binaries to zk-environment also increase reth block-time and fixes upgrade-test which improves tests stability ## Why ❔ we rely on cargo-nextest in tests ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. * feat: add da clients (matter-labs#2743) ## What ❔ This PR changes the approach to managing 3rd party DA clients. It was assumed before that they will be stored in a separate repository (hyperchain-da), but to simplify the processes and solve a recursive dependency problem, we decided to manage those within `zksync-era`. The config now defines which DA client will be used, for proto-based configuration it requires adding these lines to general.yaml: ``` da_client: avail: api_node_url: wss://turing-rpc.avail.so/ws bridge_api_url: undefined seed: SEED_PHRASE app_id: 82 timeout: 3 max_retries: 5 ``` for env-based: ``` DA_CLIENT="Avail" DA_API_NODE_URL="localhost:12345" DA_BRIDGE_API_URL="localhost:54321" DA_SEED="SEED_PHRASE" DA_APP_ID=1 DA_TIMEOUT=2 DA_MAX_RETRIES=3 ``` If no config is provided - the default behavior is to use NoDA client (same as now). The `da_client` config might be merged with `da_dispatcher` at some point as the second depends on the first one, so their separation does not make much sense (apart from simplification of the configs). But I'd prefer to do it as a separate PR in case we decide to merge them. The client was reimplemented using only lightweight libraries from crates.io, so it doesn't have any visible impact on build time. ## Why ❔ To enable seamless integration with 3rd party DA clients in `zksync-era`. ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. * feat(zk_toolbox): deploy legacy bridge (matter-labs#2837) ## What ❔ Support legacy bridges in zk toolbox ## Why ❔ We still have some legacy bridges in production, so we have to have an opportunity to test it ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. --------- Signed-off-by: Danil <deniallugo@gmail.com> * feat(zk_toolbox): Add external_node consensus support (matter-labs#2821) ## What ❔ Add external_node consensus support --------- Signed-off-by: Danil <deniallugo@gmail.com> Co-authored-by: Danil <deniallugo@gmail.com> * feat: Selector generator tool (matter-labs#2844) ## What ❔ * A small tool to generate the selector hashes based on the ABI from json files ## Why ❔ * The output json can be useful for humans to better understand some of the errors (and calldata) * It can also be read by our tools, to make the debugging easier. In the future, we could call this tool regularly on each contracts version change, but for now it can stay as manual. * feat(prover): Optimize setup keys loading (matter-labs#2847) ## What ❔ - Loads setup keys to memory in parallel (for GPU prover only). - Refactors a bunch of related code for simplicity. ## Why ❔ - Locally I've observed load time going from ~30s to ~12s, so ~60% improvement for prover start time. - Readability & maintainability. ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. * feat: Bump crypto and protocol deps (matter-labs#2825) ## What ❔ - Use latest versions of crypto, protocol, gpu, and consensus crates. - Remove solved cargo deny advisories from the allowlist. ## Why ❔ - A bunch of fixes/improvements were done. - Optimization of dependency graph. - Solving cargo deny advisories. * feat(prover): Refactor WitnessGenerator (matter-labs#2845) ## What ❔ Introduce new structure for witness generators. Introduce `ArtifactsManager` trait responsible for operations with object store and artifacts. ## Why ❔ <!-- Why are these changes done? What goal do they contribute to? What are the principles behind them? --> <!-- Example: PR templates ensure PR reviewers, observers, and future iterators are in context about the evolution of repos. --> ## Checklist <!-- Check your PR fulfills the following items. --> <!-- For draft PRs check the boxes as you complete them. --> - [ ] PR title corresponds to the body of PR (we generate changelog entries from PRs). - [ ] Tests for the changes have been added / updated. - [ ] Documentation comments have been added / updated. - [ ] Code has been formatted via `zk fmt` and `zk lint`. * Move hyperchain-da to zksync-era * Uncomment validium config checks --------- Signed-off-by: Danil <deniallugo@gmail.com> Co-authored-by: Patrick <pab@matterlabs.dev> Co-authored-by: Matías Ignacio González <maigonzalez@fi.uba.ar> Co-authored-by: Artem Fomiuk <88630083+Artemka374@users.noreply.github.com> Co-authored-by: perekopskiy <53865202+perekopskiy@users.noreply.github.com> Co-authored-by: Igor Aleksanov <popzxc@yandex.ru> Co-authored-by: Grzegorz Prusak <pompon.pompon@gmail.com> Co-authored-by: Moshe Shababo <17073733+moshababo@users.noreply.github.com> Co-authored-by: Alex Ostrovski <slowli@users.noreply.github.com> Co-authored-by: Joonatan Saarhelo <joon.saar@gmail.com> Co-authored-by: Danil <deniallugo@gmail.com> Co-authored-by: Marcin M <128217157+mm-zk@users.noreply.github.com> Co-authored-by: Dima Zhornyk <55756184+dimazhornyk@users.noreply.github.com>
1 parent 1d3a17a commit 824fa98

File tree

286 files changed

+13695
-12750
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

286 files changed

+13695
-12750
lines changed

.github/workflows/build-docker-from-tag.yml

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ concurrency: docker-build
2323
jobs:
2424
setup:
2525
name: Setup
26-
runs-on: [ubuntu-latest]
26+
runs-on: [ ubuntu-latest ]
2727
outputs:
2828
image_tag_suffix: ${{ steps.set.outputs.image_tag_suffix }}
2929
prover_fri_gpu_key_id: ${{ steps.extract-prover-fri-setup-key-ids.outputs.gpu_short_commit_sha }}
@@ -48,7 +48,7 @@ jobs:
4848
4949
build-push-core-images:
5050
name: Build and push image
51-
needs: [setup]
51+
needs: [ setup ]
5252
uses: ./.github/workflows/build-core-template.yml
5353
if: contains(github.ref_name, 'core')
5454
secrets:
@@ -60,7 +60,7 @@ jobs:
6060

6161
build-push-tee-prover-images:
6262
name: Build and push images
63-
needs: [setup]
63+
needs: [ setup ]
6464
uses: ./.github/workflows/build-tee-prover-template.yml
6565
if: contains(github.ref_name, 'core')
6666
secrets:
@@ -72,7 +72,7 @@ jobs:
7272

7373
build-push-contract-verifier:
7474
name: Build and push image
75-
needs: [setup]
75+
needs: [ setup ]
7676
uses: ./.github/workflows/build-contract-verifier-template.yml
7777
if: contains(github.ref_name, 'contract_verifier')
7878
secrets:
@@ -83,20 +83,20 @@ jobs:
8383

8484
build-push-prover-images:
8585
name: Build and push image
86-
needs: [setup]
86+
needs: [ setup ]
8787
uses: ./.github/workflows/build-prover-template.yml
8888
if: contains(github.ref_name, 'prover')
8989
with:
9090
image_tag_suffix: ${{ needs.setup.outputs.image_tag_suffix }}
9191
ERA_BELLMAN_CUDA_RELEASE: ${{ vars.ERA_BELLMAN_CUDA_RELEASE }}
92-
CUDA_ARCH: "60;70;75;89"
92+
CUDA_ARCH: "60;70;75;80;89"
9393
secrets:
9494
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
9595
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
9696

9797
build-push-witness-generator-image-avx512:
9898
name: Build and push image
99-
needs: [setup]
99+
needs: [ setup ]
100100
uses: ./.github/workflows/build-witness-generator-template.yml
101101
if: contains(github.ref_name, 'prover')
102102
with:
@@ -110,7 +110,7 @@ jobs:
110110

111111
build-gar-prover-fri-gpu:
112112
name: Build GAR prover FRI GPU
113-
needs: [setup, build-push-prover-images]
113+
needs: [ setup, build-push-prover-images ]
114114
uses: ./.github/workflows/build-prover-fri-gpu-gar.yml
115115
if: contains(github.ref_name, 'prover')
116116
with:

.github/workflows/ci-core-reusable.yml

Lines changed: 33 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ jobs:
7171
strategy:
7272
fail-fast: false
7373
matrix:
74-
vm_mode: [ "old", "new" ]
74+
vm_mode: [ "OLD", "NEW" ]
7575

7676
steps:
7777
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4
@@ -87,13 +87,12 @@ jobs:
8787
8888
- name: Loadtest configuration
8989
run: |
90-
echo EXPECTED_TX_COUNT=${{ matrix.vm_mode == 'new' && 21000 || 16000 }} >> .env
90+
echo EXPECTED_TX_COUNT=${{ matrix.vm_mode == 'NEW' && 21000 || 16000 }} >> .env
9191
echo ACCOUNTS_AMOUNT="100" >> .env
9292
echo MAX_INFLIGHT_TXS="10" >> .env
9393
echo SYNC_API_REQUESTS_LIMIT="15" >> .env
9494
echo FAIL_FAST=true >> .env
9595
echo IN_DOCKER=1 >> .env
96-
echo DATABASE_MERKLE_TREE_MODE=lightweight >> .env
9796
9897
- name: Start services
9998
run: |
@@ -107,23 +106,34 @@ jobs:
107106
ci_run git config --global --add safe.directory /usr/src/zksync/contracts/system-contracts
108107
ci_run git config --global --add safe.directory /usr/src/zksync/contracts
109108
110-
ci_run zk
111-
ci_run zk init --local-legacy-bridge-testing
109+
ci_run ./bin/zkt
110+
ci_run zk_inception chain create \
111+
--chain-name legacy \
112+
--chain-id sequential \
113+
--prover-mode no-proofs \
114+
--wallet-creation localhost \
115+
--l1-batch-commit-data-generator-mode rollup \
116+
--base-token-address 0x0000000000000000000000000000000000000001 \
117+
--base-token-price-nominator 1 \
118+
--base-token-price-denominator 1 \
119+
--set-as-default false \
120+
--ignore-prerequisites \
121+
--legacy-bridge
122+
123+
ci_run zk_inception ecosystem init --dev --verbose
124+
ci_run zk_supervisor contracts --test-contracts
112125
113126
# `sleep 60` because we need to wait until server added all the tokens
114127
- name: Run server
115128
run: |
116-
EXPERIMENTAL_VM_STATE_KEEPER_FAST_VM_MODE=${{ matrix.vm_mode }} \
117-
CHAIN_MEMPOOL_DELAY_INTERVAL=50 \
118-
PASSED_ENV_VARS="EXPERIMENTAL_VM_STATE_KEEPER_FAST_VM_MODE,CHAIN_MEMPOOL_DELAY_INTERVAL" \
119-
ci_run zk server --uring --components api,tree,eth,state_keeper,housekeeper,commitment_generator,vm_runner_protective_reads &>server.log &
129+
ci_run sed -i -e 's/mode: FULL/mode: LIGHTWEIGHT/' chains/legacy/configs/general.yaml
130+
ci_run sed -i -e 's/state_keeper_fast_vm_mode:.*/state_keeper_fast_vm_mode: ${{ matrix.vm_mode }}/' chains/legacy/configs/general.yaml
131+
ci_run sed -i -e 's/delay_interval:.*/delay_interval: 50/' chains/legacy/configs/general.yaml
132+
ci_run zk_inception server --uring --chain=legacy --components api,tree,eth,state_keeper,housekeeper,commitment_generator,vm_runner_protective_reads &>server.log &
120133
ci_run sleep 60
121134
122-
- name: Deploy legacy era contracts
123-
run: ci_run zk contract setup-legacy-bridge-era
124-
125135
- name: Perform loadtest
126-
run: ci_run zk run loadtest
136+
run: ci_run zk_supervisor t loadtest -v --chain=legacy
127137

128138
- name: Show server.log logs
129139
if: always()
@@ -293,7 +303,7 @@ jobs:
293303

294304
- name: Show revert.log logs
295305
if: always()
296-
run: ci_run cat core/tests/revert-test/revert.log || true
306+
run: ci_run cat logs/revert/default/server.log || true
297307

298308
- name: Show upgrade.log logs
299309
if: always()
@@ -382,7 +392,11 @@ jobs:
382392

383393
- name: Run revert test
384394
run: |
385-
ENABLE_CONSENSUS=${{ matrix.consensus }} DEPLOYMENT_MODE=${{ matrix.deployment_mode }} PASSED_ENV_VARS="ENABLE_CONSENSUS,DEPLOYMENT_MODE" ci_run zk test i revert-en
395+
ENABLE_CONSENSUS=${{ matrix.consensus }} \
396+
DEPLOYMENT_MODE=${{ matrix.deployment_mode }} \
397+
PASSED_ENV_VARS="ENABLE_CONSENSUS,DEPLOYMENT_MODE" \
398+
ci_run zk test i revert-en
399+
386400
# test terminates the nodes, so we restart them.
387401
if [[ "${{ matrix.deployment_mode }}" == "Rollup" ]]; then
388402
ZKSYNC_ENV=docker ci_run zk server --components=$SERVER_COMPONENTS &>>server.log &
@@ -414,13 +428,13 @@ jobs:
414428
if: always()
415429
run: ci_run cat ext-node.log || true
416430

417-
- name: Show revert_main.log logs
431+
- name: Show revert logs (main node)
418432
if: always()
419-
run: ci_run cat core/tests/revert-test/revert_main.log || true
433+
run: ci_run cat logs/revert/en/default/server.log || true
420434

421-
- name: Show revert_ext.log logs
435+
- name: Show revert logs (EN)
422436
if: always()
423-
run: ci_run cat core/tests/revert-test/revert_ext.log || true
437+
run: ci_run cat logs/revert/en/default/external_node.log || true
424438

425439
- name: Show upgrade.log logs
426440
if: always()

.github/workflows/ci-zk-toolbox-reusable.yml

Lines changed: 59 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,19 @@ on:
44

55
env:
66
CLICOLOR: 1
7+
# We run multiple binaries in parallel, and by default they will try to utilize all the
8+
# available CPUs. In tests, there is not much CPU-intensive work (rayon), but a lot of
9+
# async work (tokio), so we prioritize tokio.
10+
TOKIO_WORKER_THREADS: 4
11+
RAYON_NUM_THREADS: 2
712

813
jobs:
914
lint:
1015
name: lint
1116
uses: ./.github/workflows/ci-core-lint-reusable.yml
1217

1318
tests:
14-
runs-on: [ matterlabs-ci-runner ]
19+
runs-on: [ matterlabs-ci-runner-ultra-performance ]
1520
steps:
1621
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4
1722
with:
@@ -127,6 +132,30 @@ jobs:
127132
--prover-db-name=zksync_prover_localhost_custom_token \
128133
--port-offset 3000 \
129134
--chain custom_token
135+
136+
- name: Create and initialize Consensus chain
137+
run: |
138+
ci_run zk_inception chain create \
139+
--chain-name consensus \
140+
--chain-id sequential \
141+
--prover-mode no-proofs \
142+
--wallet-creation localhost \
143+
--l1-batch-commit-data-generator-mode validium \
144+
--base-token-address ${{ env.address }} \
145+
--base-token-price-nominator 3 \
146+
--base-token-price-denominator 2 \
147+
--set-as-default false \
148+
--ignore-prerequisites
149+
150+
ci_run zk_inception chain init \
151+
--deploy-paymaster \
152+
--l1-rpc-url=http://localhost:8545 \
153+
--server-db-url=postgres://postgres:notsecurepassword@localhost:5432 \
154+
--server-db-name=zksync_server_localhost_consensus \
155+
--prover-db-url=postgres://postgres:notsecurepassword@localhost:5432 \
156+
--prover-db-name=zksync_prover_localhost_consensus \
157+
--port-offset 4000 \
158+
--chain consensus
130159
131160
- name: Build test dependencies
132161
run: |
@@ -137,6 +166,10 @@ jobs:
137166
ci_run zk_inception server --ignore-prerequisites --chain era &> ${{ env.SERVER_LOGS_DIR }}/rollup.log &
138167
ci_run zk_inception server --ignore-prerequisites --chain validium &> ${{ env.SERVER_LOGS_DIR }}/validium.log &
139168
ci_run zk_inception server --ignore-prerequisites --chain custom_token &> ${{ env.SERVER_LOGS_DIR }}/custom_token.log &
169+
ci_run zk_inception server --ignore-prerequisites --chain consensus \
170+
--components=api,tree,eth,state_keeper,housekeeper,commitment_generator,vm_runner_protective_reads,vm_runner_bwip,vm_playground,da_dispatcher,consensus \
171+
&> ${{ env.SERVER_LOGS_DIR }}/consensus.log &
172+
140173
ci_run sleep 5
141174
142175
- name: Run integration tests
@@ -150,9 +183,13 @@ jobs:
150183
ci_run zk_supervisor test integration --no-deps --ignore-prerequisites --chain custom_token &> ${{ env.INTEGRATION_TESTS_LOGS_DIR }}/custom_token.log &
151184
PID3=$!
152185
186+
ci_run zk_supervisor test integration --no-deps --ignore-prerequisites --chain consensus &> ${{ env.INTEGRATION_TESTS_LOGS_DIR }}/consensus.log &
187+
PID4=$!
188+
153189
wait $PID1
154190
wait $PID2
155191
wait $PID3
192+
wait $PID4
156193
157194
- name: Init external nodes
158195
run: |
@@ -168,6 +205,10 @@ jobs:
168205
--db-name=zksync_en_localhost_era_custom_token --l1-rpc-url=http://localhost:8545 --chain custom_token
169206
ci_run zk_inception external-node init --ignore-prerequisites --chain custom_token
170207
208+
ci_run zk_inception external-node configs --db-url=postgres://postgres:notsecurepassword@localhost:5432 \
209+
--db-name=zksync_en_localhost_era_consensus --l1-rpc-url=http://localhost:8545 --chain consensus
210+
ci_run zk_inception external-node init --ignore-prerequisites --chain consensus
211+
171212
- name: Run recovery tests (from snapshot)
172213
run: |
173214
@@ -180,9 +221,13 @@ jobs:
180221
ci_run zk_supervisor test recovery --snapshot --no-deps --ignore-prerequisites --verbose --chain custom_token &> ${{ env.SNAPSHOT_RECOVERY_LOGS_DIR }}//custom_token.log &
181222
PID3=$!
182223
224+
ci_run zk_supervisor test recovery --snapshot --no-deps --ignore-prerequisites --verbose --chain consensus &> ${{ env.SNAPSHOT_RECOVERY_LOGS_DIR }}//consensus.log &
225+
PID4=$!
226+
183227
wait $PID1
184228
wait $PID2
185229
wait $PID3
230+
wait $PID4
186231
187232
- name: Run recovery tests (from genesis)
188233
run: |
@@ -195,15 +240,20 @@ jobs:
195240
ci_run zk_supervisor test recovery --no-deps --no-kill --ignore-prerequisites --verbose --chain custom_token &> ${{ env.GENESIS_RECOVERY_LOGS_DIR }}/custom_token.log &
196241
PID3=$!
197242
243+
ci_run zk_supervisor test recovery --no-deps --no-kill --ignore-prerequisites --verbose --chain consensus &> ${{ env.GENESIS_RECOVERY_LOGS_DIR }}/consensus.log &
244+
PID4=$!
245+
198246
wait $PID1
199247
wait $PID2
200248
wait $PID3
249+
wait $PID4
201250
202251
- name: Run external node server
203252
run: |
204253
ci_run zk_inception external-node run --ignore-prerequisites --chain era &> ${{ env.EXTERNAL_NODE_LOGS_DIR }}/rollup.log &
205254
ci_run zk_inception external-node run --ignore-prerequisites --chain validium &> ${{ env.EXTERNAL_NODE_LOGS_DIR }}/validium.log &
206255
ci_run zk_inception external-node run --ignore-prerequisites --chain custom_token &> ${{ env.EXTERNAL_NODE_LOGS_DIR }}/custom_token.log &
256+
ci_run zk_inception external-node run --ignore-prerequisites --chain consensus --enable-consensus &> ${{ env.EXTERNAL_NODE_LOGS_DIR }}/consensus.log &
207257
ci_run sleep 5
208258
209259
- name: Run integration tests en
@@ -217,9 +267,13 @@ jobs:
217267
ci_run zk_supervisor test integration --no-deps --ignore-prerequisites --external-node --chain custom_token &> ${{ env.INTEGRATION_TESTS_EN_LOGS_DIR }}/custom_token.log &
218268
PID3=$!
219269
270+
ci_run zk_supervisor test integration --no-deps --ignore-prerequisites --external-node --chain consensus &> ${{ env.INTEGRATION_TESTS_EN_LOGS_DIR }}/consensus.log &
271+
PID4=$!
272+
220273
wait $PID1
221274
wait $PID2
222275
wait $PID3
276+
wait $PID4
223277
224278
- name: Run revert tests
225279
run: |
@@ -234,10 +288,14 @@ jobs:
234288
235289
ci_run zk_supervisor test revert --no-deps --external-node --no-kill --ignore-prerequisites --chain custom_token &> ${{ env.REVERT_LOGS_DIR }}/custom_token.log &
236290
PID3=$!
291+
292+
ci_run zk_supervisor test revert --no-deps --external-node --no-kill --ignore-prerequisites --chain consensus &> ${{ env.REVERT_LOGS_DIR }}/consensus.log &
293+
PID4=$!
237294
238295
wait $PID1
239296
wait $PID2
240297
wait $PID3
298+
wait $PID4
241299
242300
243301
# Upgrade tests should run last, because as soon as they

.github/workflows/ci.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ jobs:
7676
- 'etc/**'
7777
- 'contracts/**'
7878
- 'infrastructure/zk/**'
79+
- 'docker/zk-environment/**'
7980
- '!**/*.md'
8081
- '!**/*.MD'
8182

0 commit comments

Comments
 (0)