feat(basic_witness_input_producer): Witness inputs queued after BWIP run #345

EmilLuta · 2023-10-28T16:04:44Z

What ❔

Add supporting transition state for witness_input jobs. Prior to this PR, MetadataCalculator would create witness_input jobs that are ready to be processed by WitnessGenerators. After this PR, WitnessGenerators will need to wait for BasicWitnessInputProducer to generate it's artifacts as well. This PR must be deployed after BWIP is deployed in the given environment.

Why ❔

Necessary for transitioning to Witness Generation from Blob.

Other remarks

Deployment will be rather involved. @EmilLuta will own it with support from @Deniallugo, starting 31st of Oct, 2023. Steps:

for env in ['stage2', 'testnet', 'boojnet', 'mainnet2'] {
  - Deploy to <ENV> BWIP.
  - Wait for all `witness_input` jobs to be finished prior to BWIP's processing (or mark them as manually skipped)
  - Deploy to <ENV> this patch.
  - Turn on Shadow Proving
  - Observe
}

Checklist

PR title corresponds to the body of PR (we generate changelog entries from PRs).
Tests for the changes have been added / updated.
Documentation comments have been added / updated.
Code has been formatted via zk fmt and zk lint.

core/lib/dal/src/witness_generator_dal.rs

codecov · 2023-10-31T08:54:08Z

Codecov Report

Attention: 13 lines in your changes are missing coverage. Please review.

Comparison is base (ba88f67) 35.85% compared to head (dd2fb03) 35.80%.
Report is 11 commits behind head on main.

❗ Current head dd2fb03 differs from pull request most recent head e243a40. Consider uploading reports for the commit e243a40 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #345      +/-   ##
==========================================
- Coverage   35.85%   35.80%   -0.05%     
==========================================
  Files         519      519              
  Lines       27828    27841      +13     
==========================================
- Hits         9978     9969       -9     
- Misses      17850    17872      +22

Files	Coverage Δ
core/lib/dal/src/witness_generator_dal.rs	`48.37% <0.00%> (-1.39%)`	⬇️
...ksync_core/src/basic_witness_input_producer/mod.rs	`0.00% <0.00%> (ø)`

... and 100 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🤖 I have created a release *beep* *boop* --- ## [18.0.0](core-v17.1.0...core-v18.0.0) (2023-11-14) ### ⚠ BREAKING CHANGES * boojum integration ([#112](#112)) ### Features * **basic_witness_input_producer:** Witness inputs queued after BWIP run ([#345](#345)) ([9c2be91](9c2be91)) * boojum integration ([#112](#112)) ([e76d346](e76d346)) * **core:** adds a get proof endpoint in zks namespace ([#455](#455)) ([f4313a4](f4313a4)) * **core:** Split config definitions and deserialization ([#414](#414)) ([c7c6b32](c7c6b32)) * **dal:** Do not load config from env in DAL crate ([#444](#444)) ([3fe1bb2](3fe1bb2)) * **house_keeper:** Remove GCS Blob Cleaner ([#321](#321)) ([9548914](9548914)) * **job-processor:** report attempts metrics ([#448](#448)) ([ab31f03](ab31f03)) * **vm:** Use the one interface for all vms ([#277](#277)) ([91bb99b](91bb99b)) ### Bug Fixes * **boojnet:** various boojnet fixes ([#462](#462)) ([f13648c](f13648c)) * change vks upgrade logic ([#491](#491)) ([cb394f3](cb394f3)) * **eth-sender:** Correct ABI for get_verification_key ([#445](#445)) ([8af0d85](8af0d85)) * **metadata-calculator:** Save commitment for pre-boojum ([#481](#481)) ([664ce33](664ce33)) * Versioned L1 batch metadata ([#450](#450)) ([8a40dc3](8a40dc3)) * **vm:** storage_refunds for `vm_refunds_enhancement` ([#449](#449)) ([1e1e59f](1e1e59f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

EmilLuta requested a review from a team as a code owner October 28, 2023 16:04

popzxc reviewed Oct 30, 2023

View reviewed changes

core/lib/dal/src/witness_generator_dal.rs Show resolved Hide resolved

RomanBrodetski approved these changes Oct 30, 2023

View reviewed changes

popzxc approved these changes Oct 30, 2023

View reviewed changes

Deniallugo approved these changes Oct 30, 2023

View reviewed changes

feat(basic_witness_input_producer): Witness inputs queued after BWIP run

dd2fb03

EmilLuta force-pushed the evl-add-new-state-machine-to-witness-transition branch from f111191 to dd2fb03 Compare October 31, 2023 08:39