Update sequencer 2 (#26)

* chore(starknet_batcher): delete obsolete todo (starkware-libs#2389) * chore(blockifier): add global max_sierra_gas to versioned constants (starkware-libs#2330) * feat(starknet_api): checked mul for gas vector (starkware-libs#2300) * feat(consensus): proposer rotates across validators (starkware-libs#2405) * feat(sequencing): validate streamed proposals (starkware-libs#2305) * feat(ci): deny rust warnings in all workflows (starkware-libs#2388) Signed-off-by: Dori Medini <dori@starkware.co> * feat(blockifier): compute allocation cost (starkware-libs#2301) * chore(starknet_sequencer_infra): add dynamic logging util fn commit-id:9ffe9fbe * chore(starknet_sequencer_infra): add tracing test commit-id:76d16e9a * chore(starknet_sequencer_infra): add run_until utility fn commit-id:194a4b6c * chore(infra_utils): change run_until to support async functions commit-id:92e1f8a3 * chore(starknet_integration_tests): use run_until to await for block creation commit-id:667e001c * chore(infra_utils): rename logger struct commit-id:6520ae54 * chore(blockifier): explicit creation of AccountTransaction (starkware-libs#2331) * test(starknet_integration_tests): test commit blocks by running multiple heights * chore(starknet_batcher): set temp gas prices in propose block input (starkware-libs#2341) * chore(starknet_batcher): set use_kzg_da flag in build block input (starkware-libs#2345) * feat(ci): inherit the rust toolchain toml in moonrepo action (starkware-libs#2423) Signed-off-by: Dori Medini <dori@starkware.co> * chore(blockifier): enforce_fee() impl by api::executable_transaction::AccountTransaction (starkware-libs#2377) * chore(starknet_integration_tests): inherit sequencer node's stdout (starkware-libs#2427) * chore(blockifier): invoke() declare() deploy_account() change ret val to api_tx (starkware-libs#2412) * chore(starknet_consensus_manager): set proposer address in propose block input (starkware-libs#2346) * feat(consensus): add observer mode * feat(consensus): sequencer context broadcasts votes (starkware-libs#2422) * chore(deployment): support unified deployment config (starkware-libs#2378) * feat(starknet_api): add sierra version to class info (starkware-libs#2313) * refactor(starknet_api): change default sierra contract class to valid one (starkware-libs#2439) * feat(starknet_l1_provider): add uninitiailized state and make it the default (starkware-libs#2434) This is to comply with upcoming integration with infra, which separates instantiation with initialization. In particular, `Pending` state should be already post-syncing with L1, whereas `Uninitialized` is unsynced and unusable. Co-Authored-By: Gilad Chase <gilad@starkware.com> * feat(papyrus_storage)!: bump storage version for version 13.4 (starkware-libs#2333) * feat(native_blockifier): allow deserialization of python l1_data_gas (starkware-libs#2447) * refactor(blockifier): split FC to groups base on their tags (starkware-libs#2236) * test(consensus): remove warning on into mock propsal part (starkware-libs#2448) * chore(blockifier): use test_utils::invoke_tx() instead of trans::test_utils::account_invoke_tx() (starkware-libs#2428) * chore(blockifier): save sierra to Feature contracts (starkware-libs#2370) * feat(blockifier): don't count Sierra gas in CairoSteps mode (starkware-libs#2440) * chore(blockifier): convert Sierra gas to L1 gas if in L1 gas mode (starkware-libs#2451) * feat(blockifier): add comprehensive state diff versioned constant (starkware-libs#2407) * chore(starknet_consensus_manager): add chain_id to config * refactor(papyrus_p2p_sync): add random_header utility function (starkware-libs#2381) * chore(starknet_batcher): pass block info from consensus to batcher (starkware-libs#2238) * test(starknet_mempool): tx added to mempool are forwarded to propagator client (starkware-libs#2288) * fix: fix CR comments * test(starknet_mempool): tx added to mempool are forwarded to propagator client * feat(sequencing): cache proposals from bigger heights(starkware-libs#2325) * fix: change to latest ubuntu version in feature combo CI (starkware-libs#2414) * chore(blockifier): replace entry_point_gas_cost with initial_budget (starkware-libs#2247) * test(starknet_gateway): handle todo in test_get_block_info (starkware-libs#2267) * chore(starknet_api): revert use get_packaget_dir instead of env var This reverts commit c45f5cc. commit-id:a48736e7 * chore(starknet_api): rely on env::current_dir() instead of CARGO_MANIFEST_DIR commit-id:301ed4eb * chore(blockifier): move env var from run time to compile time commit-id:80a7265d * chore(starknet_sierra_compile): move env var from run time to compile time commit-id:6e7f2a75 * chore: remove the use of zero as a validator id (starkware-libs#2411) * refactor(papyrus_p2p_sync): add_test receives query size instead of constant (starkware-libs#2379) * fix(blockifier): merge state diff with squash (starkware-libs#2310) * feat(blockifier): get revert receipt only in case of revert (starkware-libs#2471) * chore(starknet_integration_tests): create chain info once (starkware-libs#2482) * chore(starknet_sierra_compile): split build utils commit-id:0f504fd7 * chore(starknet_sierra_compile): set runtime-accessible out_dir env var commit-id:539f16db * chore(starknet_sierra_compile): avoid using OUT_DIR in run time commit-id:cd6fba29 * refactor(starknet_api): use const in sierra version (starkware-libs#2477) * chore: cleanups of OUT_DIR env var (starkware-libs#2484) commit-id:18d61b1d * chore(starknet_api): shorten executable_transaction usage path * fix(sequencing): remove old proposal pipes from consensus (starkware-libs#2452) * test(starknet_integration_tests): match sequencer address with default validator id (starkware-libs#2486) * fix(ci): move specific versioned deps to root toml (starkware-libs#2487) * fix(ci): move specific versioned deps to root toml Signed-off-by: Dori Medini <dori@starkware.co> * fix(starknet_sierra_compile): fix build.rs Signed-off-by: Dori Medini <dori@starkware.co> * chore(starknet_batcher): in block builder use the consensus suplied sequncer address (starkware-libs#2409) * chore: cleanups of CARGO_MANIFEST_DIR env var commit-id:c96f2d88 * fix(starknet_sierra_compile): missing import in feature (starkware-libs#2495) commit-id:abd0a286 * chore(blockifier): add keccak_builtin_gas_cost (starkware-libs#2327) * chore(blockifier): add struct ExecutionFlags to AccTx & use instead of transaction::ExecutionFlags (starkware-libs#2429) * chore(blockifier): add new constructor to AccountTransaction (starkware-libs#2431) * chore(blockifier): remove only_qury from IvokeTxArgs (starkware-libs#2437) * chore(blockifier): remove struct ExecutionFlags and replace w concurrency_mode bool (starkware-libs#2470) * chore(blockifier): remove declare.rs deploy_account.rs invoke.rs from blockifier (starkware-libs#2478) --------- Signed-off-by: Dori Medini <dori@starkware.co> Co-authored-by: YaelD <70628564+Yael-Starkware@users.noreply.github.com> Co-authored-by: aner-starkware <147302140+aner-starkware@users.noreply.github.com> Co-authored-by: yoavGrs <97383386+yoavGrs@users.noreply.github.com> Co-authored-by: matan-starkware <97523054+matan-starkware@users.noreply.github.com> Co-authored-by: guy-starkware <guy.n@starkware.co> Co-authored-by: dorimedini-starkware <dori@starkware.co> Co-authored-by: Itay Tsabary <itayt@starkware.co> Co-authored-by: avivg-starkware <aviv.g@starkware.co> Co-authored-by: Yair Bakalchuk <yair@starkware.co> Co-authored-by: Arnon Hod <arnon@starkware.co> Co-authored-by: Alon Haramati <91828241+alonh5@users.noreply.github.com> Co-authored-by: Asmaa Magdoub <asmaa@starkware.co> Co-authored-by: alon-dotan-starkware <alon.dotan@starkware.co> Co-authored-by: AvivYossef-starkware <141143145+AvivYossef-starkware@users.noreply.github.com> Co-authored-by: giladchase <gilad@starkware.co> Co-authored-by: Gilad Chase <gilad@starkware.com> Co-authored-by: DvirYo-starkware <115620476+DvirYo-starkware@users.noreply.github.com> Co-authored-by: nimrod-starkware <143319383+nimrod-starkware@users.noreply.github.com> Co-authored-by: Meshi Peled <141231558+meship-starkware@users.noreply.github.com> Co-authored-by: Yoni <78365039+Yoni-Starkware@users.noreply.github.com> Co-authored-by: Yael Doweck <yael@starkware.co> Co-authored-by: ShahakShama <70578257+ShahakShama@users.noreply.github.com> Co-authored-by: Alon-Lukatch-Starkware <alon.l@starkware.co> Co-authored-by: Yonatan-Starkware <yonatan.k@starkware.co> Co-authored-by: Yair <92672946+yair-starkware@users.noreply.github.com> Co-authored-by: Itay-Tsabary-Starkware <106665835+Itay-Tsabary-Starkware@users.noreply.github.com>
lambdaclass · Dec 6, 2024 · 64cebe2 · 64cebe2
1 parent bfc5b6b
commit 64cebe2
Show file tree

Hide file tree

Showing 170 changed files with 24,792 additions and 1,756 deletions.
diff --git a/.github/actions/install_rust/action.yml b/.github/actions/install_rust/action.yml
@@ -7,3 +7,4 @@ runs:
     - uses: moonrepo/setup-rust@v1
       with:
         cache-base: main(-v[0-9].*)?
+        inherit-toolchain: true
diff --git a/.github/workflows/blockifier_ci.yml b/.github/workflows/blockifier_ci.yml
@@ -34,14 +34,17 @@ on:
       - 'scripts/install_build_tools.sh'
       - 'scripts/sequencer-ci.Dockerfile'
 
+env:
+  RUSTFLAGS: "-D warnings"
+
 # On PR events, cancel existing CI runs on this same PR for this workflow.
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}-${{ github.job }}
   cancel-in-progress: ${{ github.event_name == 'pull_request' }}
 
 jobs:
   feature-combo-builds:
-    runs-on: starkware-ubuntu-20-04-medium
+    runs-on: starkware-ubuntu-latest-medium
     steps:
       - uses: actions/checkout@v4
         with:
@@ -57,5 +60,4 @@ jobs:
       - run: cargo test -p blockifier --features transaction_serde
       # cairo_native is not activated by any workspace crate; test the build.
       - run: cargo build -p blockifier --features cairo_native
-      # TODO(Meshi): Reinstate native tests once it works again...
-      # - run: cargo test -p blockifier --features cairo_native
+      - run: cargo test -p blockifier --features cairo_native
diff --git a/.github/workflows/blockifier_compiled_cairo.yml b/.github/workflows/blockifier_compiled_cairo.yml
@@ -15,6 +15,9 @@ on:
       - 'crates/blockifier/tests/requirements.txt'
       - 'scripts/dependencies.sh'
 
+env:
+  RUSTFLAGS: "-D warnings"
+
 # On PR events, cancel existing CI runs on this same PR for this workflow.
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}-${{ github.job }}

diff --git a/.github/workflows/blockifier_reexecution_ci.yml b/.github/workflows/blockifier_reexecution_ci.yml
@@ -19,6 +19,9 @@ on:
       - 'scripts/install_build_tools.sh'
       - 'scripts/sequencer-ci.Dockerfile'
 
+env:
+  RUSTFLAGS: "-D warnings"
+
 # On PR events, cancel existing CI runs on this same PR for this workflow.
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}-${{ github.job }}

diff --git a/.github/workflows/committer_ci.yml b/.github/workflows/committer_ci.yml
@@ -18,6 +18,9 @@ on:
       - 'crates/starknet_patricia/**'
       - 'scripts/dependencies.sh'
 
+env:
+  RUSTFLAGS: "-D warnings"
+
 # On PR events, cancel existing CI runs on this same PR for this workflow.
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}-${{ github.job }}

diff --git a/.github/workflows/committer_cli_push.yml b/.github/workflows/committer_cli_push.yml
@@ -34,6 +34,9 @@ on:
       - 'crates/starknet_patricia/**'
       - 'scripts/dependencies.sh'
 
+env:
+  RUSTFLAGS: "-D warnings"
+
 # On PR events, cancel existing CI runs on this same PR for this workflow.
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}-${{ github.job }}

diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml
@@ -18,6 +18,7 @@ on:
 
 env:
   CI: 1
+  RUSTFLAGS: "-D warnings"
 
 # On PR events, cancel existing CI runs on this same PR for this workflow.
 concurrency:

diff --git a/.github/workflows/main_pr.yml b/.github/workflows/main_pr.yml
@@ -9,6 +9,9 @@ on:
       - auto_merge_enabled
       - edited
 
+env:
+  RUSTFLAGS: "-D warnings"
+
 # On PR events, cancel existing CI runs on this same PR for this workflow.
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}-${{ github.job }}

diff --git a/.github/workflows/papyrus_ci.yml b/.github/workflows/papyrus_ci.yml
@@ -22,6 +22,9 @@ on:
   merge_group:
     types: [checks_requested]
 
+env:
+  RUSTFLAGS: "-D warnings"
+
 # On PR events, cancel existing CI runs on this same PR for this workflow.
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}-${{ github.job }}

diff --git a/.github/workflows/papyrus_docker-publish.yml b/.github/workflows/papyrus_docker-publish.yml
@@ -28,6 +28,7 @@ concurrency:
 env:
   REGISTRY: ghcr.io
   REPO_NAME: ${{ github.repository }}
+  RUSTFLAGS: "-D warnings"
 
 jobs:
   docker-build-push:

diff --git a/.github/workflows/papyrus_nightly-tests-call.yml b/.github/workflows/papyrus_nightly-tests-call.yml
@@ -16,6 +16,9 @@ on:
       SLACK_ALERT_CHANNEL:
         required: true
 
+env:
+  RUSTFLAGS: "-D warnings"
+
 jobs:
   GW-integration-test-call:
     runs-on: ${{ inputs.os }}

diff --git a/.github/workflows/papyrus_nightly-tests.yml b/.github/workflows/papyrus_nightly-tests.yml
@@ -6,6 +6,9 @@ on:
     - cron: '30 0 * * *'  # Uses macos runner.
   workflow_dispatch:  # Uses ubuntu runner.
 
+env:
+  RUSTFLAGS: "-D warnings"
+
 jobs:
   GW-integration-test-ubuntu:
     uses: ./.github/workflows/papyrus_nightly-tests-call.yml

diff --git a/.github/workflows/upload_artifacts_workflow.yml b/.github/workflows/upload_artifacts_workflow.yml
@@ -5,6 +5,9 @@ on:
     workflows: [Trigger-Workflow]
     types: [completed]
 
+env:
+  RUSTFLAGS: "-D warnings"
+
 jobs:
   native-blockifier-artifacts-push:
     runs-on: starkware-ubuntu-20-04-medium

diff --git a/.github/workflows/verify-deps.yml b/.github/workflows/verify-deps.yml
@@ -4,6 +4,9 @@ on:
   schedule:
     - cron: '0 0 * * *' # Runs at 00:00 UTC every day
 
+env:
+  RUSTFLAGS: "-D warnings"
+
 jobs:
   latest_deps:
     name: Latest Dependencies

diff --git a/Cargo.lock b/Cargo.lock
diff --git a/Cargo.toml b/Cargo.toml
@@ -114,6 +114,7 @@ criterion = "0.5.1"
 deadqueue = "0.2.4"
 defaultmap = "0.5.0"
 derive_more = "0.99.17"
+enum-as-inner = "0.6.1"
 enum-assoc = "1.1.0"
 enum-iterator = "1.4.1"
 ethers = "2.0.3"
@@ -125,6 +126,7 @@ futures = "0.3.21"
 futures-channel = "0.3.21"
 futures-util = "0.3.21"
 glob = "0.3.1"
+google-cloud-storage = "0.22.1"
 goose = "0.17.0"
 hex = "0.4.3"
 http = "0.2.8"

diff --git a/config/papyrus/default_config.json b/config/papyrus/default_config.json
@@ -79,6 +79,11 @@
     "privacy": "TemporaryValue",
     "value": true
   },
+  "consensus.chain_id": {
+    "description": "The chain id of the Starknet chain.",
+    "pointer_target": "chain_id",
+    "privacy": "Public"
+  },
   "consensus.consensus_delay": {
     "description": "Delay (seconds) before starting consensus to give time for network peering.",
     "privacy": "Public",
@@ -197,7 +202,7 @@
   "consensus.validator_id": {
     "description": "The validator id of the node.",
     "privacy": "Public",
-    "value": "0x0"
+    "value": "0x64"
   },
   "monitoring_gateway.collect_metrics": {
     "description": "If true, collect and return metrics in the monitoring gateway.",
@@ -509,4 +514,4 @@
     "privacy": "Public",
     "value": true
   }
-}
+}
diff --git a/config/sequencer/default_config.json b/config/sequencer/default_config.json
@@ -104,21 +104,11 @@
     "privacy": "Public",
     "value": 0
   },
-  "batcher_config.block_builder_config.sequencer_address": {
-    "description": "The address of the sequencer.",
-    "pointer_target": "sequencer_address",
-    "privacy": "Public"
-  },
   "batcher_config.block_builder_config.tx_chunk_size": {
     "description": "The size of the transaction chunk.",
     "privacy": "Public",
     "value": 100
   },
-  "batcher_config.block_builder_config.use_kzg_da": {
-    "description": "Indicates whether the kzg mechanism is used for data availability.",
-    "privacy": "Public",
-    "value": true
-  },
   "batcher_config.block_builder_config.versioned_constants_overrides.invoke_tx_max_n_steps": {
     "description": "Maximum number of steps the invoke function is allowed to run.",
     "pointer_target": "versioned_constants_overrides.invoke_tx_max_n_steps",
@@ -205,7 +195,7 @@
     "value": "StateOnly"
   },
   "chain_id": {
-    "description": "A required param! The chain to follow.",
+    "description": "A required param! The chain to follow. For more details see https://docs.starknet.io/documentation/architecture_and_concepts/Blocks/transactions/#chain-id.",
     "param_type": "String",
     "privacy": "TemporaryValue"
   },
@@ -614,6 +604,11 @@
     "privacy": "Public",
     "value": "0.0.0.0:8080"
   },
+  "consensus_manager_config.consensus_config.chain_id": {
+    "description": "The chain id of the Starknet chain.",
+    "pointer_target": "chain_id",
+    "privacy": "Public"
+  },
   "consensus_manager_config.consensus_config.consensus_delay": {
     "description": "Delay (seconds) before starting consensus to give time for network peering.",
     "privacy": "Public",
@@ -732,7 +727,7 @@
   "consensus_manager_config.consensus_config.validator_id": {
     "description": "The validator id of the node.",
     "privacy": "Public",
-    "value": "0x0"
+    "value": "0x64"
   },
   "eth_fee_token_address": {
     "description": "A required param! Address of the ETH fee token.",
@@ -949,11 +944,6 @@
     "privacy": "Public",
     "value": ""
   },
-  "sequencer_address": {
-    "description": "A required param! The sequencer address.",
-    "param_type": "String",
-    "privacy": "TemporaryValue"
-  },
   "strk_fee_token_address": {
     "description": "A required param! Address of the STRK fee token.",
     "param_type": "String",

diff --git a/config/sequencer/presets/config-batcher.json b/config/sequencer/presets/config-batcher.json
@@ -4,5 +4,4 @@
   "strk_fee_token_address": "0x7",
   "batcher_config.storage.db_config.path_prefix": "/data",
   "batcher_config.storage.db_config.enforce_file_exists": false,
-  "sequencer_address": "0x1"
 }
-Original file line number
+Diff line change
@@ Expand Up / @@ -18,6 +18,7 @@ on: @@
     env:
       CI: 1
+      RUSTFLAGS: "-D warnings"
     # On PR events, cancel existing CI runs on this same PR for this workflow.
     concurrency:
@@ Expand Down @@