katana-executor rewrite + starknet_in_rust intergration + saya …

…namespace (starkware-libs#1697)

* refactor(katana): change stored compiled class (starkware-libs#1559)

Both `blockifier` and `starknet_in_rust` use their own proprietary contract class types. So need to have a type where the types from those two crates can be derived from.

* refactor(katana-primitives): move class related types to a separate module (starkware-libs#1560)

* Katana executor traits (starkware-libs#1561)

* feat(katana-executor): add `blockifier` executor implementations (starkware-libs#1562)

implement the executor traits introduced in starkware-libs#1561 based on `blockifier`

* Refactor Katana to using the new executor traits (starkware-libs#1563)

integrate the new executor traits introduced in starkware-libs#1561 to katana

* feat(katana-executor): add starknet_in_rust executor (starkware-libs#1564)

* doc(katana-executor): add README

* test(katana-executor): add test for transaction simulation (starkware-libs#1593)

* test(katana-executor): refactor simulate tests (starkware-libs#1606)

refactor tests

* refactor(katana): decouple `katana-primitives` from any executor impls (starkware-libs#1637)

* test(katana-executor): improve tests and add test for state (starkware-libs#1649)

- add tests for each executor's CachedState
- improve test in executor.rs
- fix sir not enough gas error
- fix test in simulate.rs

* feat(saya): load data from Katana to run `SNOS` (starkware-libs#1535)

* chore: bump dependencies

* fix: update cairo-lang

* fix: add missing run_profiler

* fix: fmt

* update cairo-vm rev

* wip

* wip

* fmt

* fix: ensure semantics analyzer plugin is registered

* chore(katana): make ethers non optional since U256 is used outside messaging

* fix: run CAIRO_FIX_TESTS

* fix(dojo-core): remove compiler warnings

* git: add sozo generated bindings to gitignore

* fix: fix class hash

* fix: class hash generated code

* fix(dojo-core): fix tests and compilation warnings

* chore: integrate snos on cairo2.5

* chore: update Cargo.lock

* wip: work on integration of katana types

* wip: fetch strategy

* fix

* feat: add parsing of state diff for genesis block

* wip

* wip

* fix: rework provider for saya and RPC implem

* wip

* wip: snos types

* wip txs

* wip: fix snos parsing need to complete inputs

* chore: switch back to git snos

* chore: bump blockifier to use Serde on TransactionExecutionInfo

* feat: add saya RPC namespace

* feat: add transaction execution info to in memory db

* fix: remove unused code covered by other test

* fix: add logic for small requests to get transactions executions info

* fix: fetch transactions executions info instead of running blockifier

* fix: fmt and clippy

* docs: fix unused type

* fix: add tests for new transactions execution retrieval

* fix: add correct test to check no pending support

* fix: restore unchanged file

* fix: rename new provider trait to TransactionTraceProvider

* refactor(katana): change stored compiled class (starkware-libs#1559)

Both `blockifier` and `starknet_in_rust` use their own proprietary contract class types. So need to have a type where the types from those two crates can be derived from.

* refactor(katana-primitives): move class related types to a separate module (starkware-libs#1560)

* Katana executor traits (starkware-libs#1561)

* feat(katana-executor): add `blockifier` executor implementations (starkware-libs#1562)

implement the executor traits introduced in starkware-libs#1561 based on `blockifier`

* Refactor Katana to using the new executor traits (starkware-libs#1563)

integrate the new executor traits introduced in starkware-libs#1561 to katana

* feat(katana-executor): add starknet_in_rust executor (starkware-libs#1564)

* doc(katana-executor): add README

* test(katana-executor): add test for transaction simulation (starkware-libs#1593)

* test(katana-executor): refactor simulate tests (starkware-libs#1606)

refactor tests

* adjust new types for Katana blockchain and SNOS

* fix: clippy

* refactor(katana): change stored compiled class (starkware-libs#1559)

Both `blockifier` and `starknet_in_rust` use their own proprietary contract class types. So need to have a type where the types from those two crates can be derived from.

* refactor(katana-primitives): move class related types to a separate module (starkware-libs#1560)

* Katana executor traits (starkware-libs#1561)

* feat(katana-executor): add `blockifier` executor implementations (starkware-libs#1562)

implement the executor traits introduced in starkware-libs#1561 based on `blockifier`

* Refactor Katana to using the new executor traits (starkware-libs#1563)

integrate the new executor traits introduced in starkware-libs#1561 to katana

* feat(katana-executor): add starknet_in_rust executor (starkware-libs#1564)

* doc(katana-executor): add README

* test(katana-executor): add test for transaction simulation (starkware-libs#1593)

* test(katana-executor): refactor simulate tests (starkware-libs#1606)

refactor tests

* feat: add new primitive for trace and blockifier conversions

* feat: add TransactionExecutionInfo convert from blockifier

* feat: add executor trait method to retrieve the execution info

* fix: ensure tx execution info is passed in block producer

* chore: restore dependency on blockifier as we use TxExecInfo primitive

* refactor(katana): decouple `katana-primitives` from any executor impls (starkware-libs#1637)

* fix: clippy / fmt

* merge

* fix: clippy

---------

Co-authored-by: Kariy <ammr.arf@gmail.com>
Co-authored-by: Ammar Arif <evergreenkary@gmail.com>

* feat(katana-primitives): add fee info type

* Refactor executor output

* fix(katana): add missing contract address in `CallInfo`

* Remove `native` feature for now

* Limit nextest build jobs

* refactor(katana-executor): add logs for transaction events and resources

* fix(katana-executor): return reverted tx as error in estimate fee

---------

Co-authored-by: glihm <dev@glihm.net>

.devcontainer/Dockerfile

@@ -4,12 +4,23 @@
 
 ARG VARIANT
 FROM mcr.microsoft.com/vscode/devcontainers/rust:${VARIANT}
+ARG VARIANT
 
 # Install additional packages
 RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
- && apt-get -y install --no-install-recommends protobuf-compiler libprotobuf-dev libclang-dev
+ && apt-get -y install --no-install-recommends protobuf-compiler libprotobuf-dev libclang-dev libzstd-dev
+
+RUN apt install -y gh libgmp3-dev software-properties-common
+
+# Install Cairo Native dependencies
+RUN curl -s https://apt.llvm.org/llvm-snapshot.gpg.key | tee /etc/apt/trusted.gpg.d/apt.llvm.org.asc > /dev/null
+RUN echo "deb http://apt.llvm.org/${VARIANT}/ llvm-toolchain-${VARIANT}-17 main" | tee /etc/apt/sources.list.d/llvm.list && apt-get update
+RUN apt-get -y install -t llvm-toolchain-${VARIANT}-17 llvm-17 llvm-17-dev llvm-17-runtime clang-17 clang-tools-17 lld-17 libpolly-17-dev libmlir-17-dev mlir-17-tools
 
-RUN apt install -y gh libgmp3-dev
+# To build Katana with 'native' feature, we need to set the following environment variables
+ENV MLIR_SYS_170_PREFIX=/usr/lib/llvm-17
+ENV LLVM_SYS_170_PREFIX=/usr/lib/llvm-17
+ENV TABLEGEN_170_PREFIX=/usr/lib/llvm-17
 
 # To allow independent workflow of the container, the rust-toolchain is explicitely given.
 RUN echo "1.74.0" > rust_toolchain_version

.github/workflows/ci.yml

@@ -14,11 +14,15 @@ jobs:
  test:
  runs-on: ubuntu-latest-16-cores
  container:
- image: ghcr.io/dojoengine/dojo-dev:5d61184
+ image: nondeterministickari/dojo-dev:cc6554da
  steps:
  - uses: actions/checkout@v3
  - uses: Swatinem/rust-cache@v2
- - run: cargo build -r --bin katana && cargo llvm-cov nextest --all-features --lcov --output-path lcov.info
+ - run: | 
+ cargo llvm-cov nextest --no-report --all-features --workspace --exclude katana --build-jobs 10
+ cargo llvm-cov nextest --no-report -p katana
+ cargo llvm-cov nextest --no-report -p katana --no-default-features --features sir
+ cargo llvm-cov report --lcov --output-path lcov.info
  - uses: codecov/codecov-action@v3
  with:
  token: ${{ secrets.CODECOV_TOKEN }}
@@ -27,7 +31,7 @@ jobs:
  ensure-wasm:
  runs-on: ubuntu-latest
  container:
- image: ghcr.io/dojoengine/dojo-dev:5d61184
+ image: nondeterministickari/dojo-dev:cc6554da
  steps:
  - uses: actions/checkout@v3
  - uses: Swatinem/rust-cache@v2
@@ -62,7 +66,7 @@ jobs:
  dojo-core-test:
  runs-on: ubuntu-latest
  container:
- image: ghcr.io/dojoengine/dojo-dev:5d61184
+ image: nondeterministickari/dojo-dev:cc6554da
  steps:
  - uses: actions/checkout@v3
  - uses: Swatinem/rust-cache@v2
@@ -71,7 +75,7 @@ jobs:
  dojo-spawn-and-move-example-test:
  runs-on: ubuntu-latest
  container:
- image: ghcr.io/dojoengine/dojo-dev:5d61184
+ image: nondeterministickari/dojo-dev:cc6554da
  steps:
  - uses: actions/checkout@v3
  - uses: Swatinem/rust-cache@v2
@@ -80,7 +84,7 @@ jobs:
  dojo-world-bindings-check:
  runs-on: ubuntu-latest
  container:
- image: ghcr.io/dojoengine/dojo-dev:5d61184
+ image: nondeterministickari/dojo-dev:cc6554da
  steps:
  - uses: actions/checkout@v3
  - uses: Swatinem/rust-cache@v2
@@ -89,7 +93,7 @@ jobs:
  clippy:
  runs-on: ubuntu-latest
  container:
- image: ghcr.io/dojoengine/dojo-dev:5d61184
+ image: nondeterministickari/dojo-dev:cc6554da
  steps:
  - uses: actions/checkout@v3
  - uses: Swatinem/rust-cache@v2
@@ -98,7 +102,7 @@ jobs:
  fmt:
  runs-on: ubuntu-latest
  container:
- image: ghcr.io/dojoengine/dojo-dev:5d61184
+ image: nondeterministickari/dojo-dev:cc6554da
  steps:
  - uses: actions/checkout@v3
  - uses: Swatinem/rust-cache@v2
@@ -107,7 +111,7 @@ jobs:
  docs:
  runs-on: ubuntu-latest
  container:
- image: ghcr.io/dojoengine/dojo-dev:5d61184
+ image: nondeterministickari/dojo-dev:cc6554da
  steps:
  - uses: actions/checkout@v3
  - uses: Swatinem/rust-cache@v2