Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(main): add eigen client extra features #317

Closed
wants to merge 43 commits into from

Conversation

juan518munoz
Copy link

@juan518munoz juan518munoz commented Oct 31, 2024

What ❔

Add extra functionality to the eigen client:

Why ❔

Checklist

  • PR title corresponds to the body of PR (we generate changelog entries from PRs).
  • Tests for the changes have been added / updated.
  • Documentation comments have been added / updated.
  • Code has been formatted via zkstack dev fmt and zkstack dev lint.

perekopskiy and others added 28 commits November 1, 2024 09:17
## What ❔

add compilers 1.5.7 to contract verifier

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.
This PR adds the ability to use `block.timestamp` in custom AA
contracts. AAs will still not have direct access to `block.timestamp`,
but can utilize it via a proxy that enforces certain constraints. The PR
introduces a `TimestampAsserter` contract that is deployed on every
chain to the user space, similar to `Multicall3`. This contract has a
single function, `assertTimestampInRange(start, end)`, which can be used
by AAs at their discretion.

The `TimestampAsserter` contract ensures that `block.timestamp` falls
within the specified `(start, end)` range. Additionally, the sequencer
verifies that the `block.timestamp` is sufficiently far from the range’s
end. This is to prevent DoS attacks where transactions pass validation
but get stuck in the mempool during execution. This constraint is
configurable and can be adjusted without requiring protocol update.

The PR also introduces two new fields to the `transactions` table:
`timestamp_asserter_range_start` and `timestamp_asserter_range_end`.
These fields are extracted during transaction execution in the sandbox
by the `ValidationTracer`. If multiple assertions are made in a single
transaction, the system captures the maximum of the starts and the
minimum of the ends, resulting in the narrowest possible time range.

Transactions with time range constraints will undergo additional
verification before being included in a block. If the current time falls
outside the transaction’s specified time range, the transaction will be
rejected with an appropriate message.

Sister PR in `era-contracts`:
matter-labs/era-contracts#843

---------

Signed-off-by: Danil <deniallugo@gmail.com>
Co-authored-by: Danil <deniallugo@gmail.com>
Makes validation tracer nicer to implement.
Implement non auth blob dispersal for eigen client

---------

Co-authored-by: Gianbelinche <39842759+gianbelinche@users.noreply.github.com>
feat(eigen-client-extra-features): get inclusion data
…3189)

## What ❔

Brushes up contract verifier code.

## Why ❔

Would make it easier to support EVM emulation in the contract verifier.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.
🤖 I have created a release *beep* *boop*
---


##
[25.1.0](matter-labs/zksync-era@core-v25.0.0...core-v25.1.0)
(2024-11-04)


### Features

* add `block.timestamp` asserter for AA
([matter-labs#3031](matter-labs#3031))
([069d38d](matter-labs@069d38d))
* allow vm2 tracers to stop execution
([matter-labs#3183](matter-labs#3183))
([9dae839](matter-labs@9dae839))
* **api:** get rid of tx receipt root
([matter-labs#3187](matter-labs#3187))
([6c034f6](matter-labs@6c034f6))
* **api:** Integrate new VM into API server (no tracers)
([matter-labs#3033](matter-labs#3033))
([8e75d4b](matter-labs@8e75d4b))
* base token integration tests
([matter-labs#2509](matter-labs#2509))
([8db7e93](matter-labs@8db7e93))
* **consensus:** enabled syncing pregenesis blocks over p2p
([matter-labs#3192](matter-labs#3192))
([6adb224](matter-labs@6adb224))
* **da-clients:** add Celestia client
([matter-labs#2983](matter-labs#2983))
([d88b875](matter-labs@d88b875))
* **da-clients:** add EigenDA client
([matter-labs#3155](matter-labs#3155))
([5161eed](matter-labs@5161eed))
* gateway preparation
([matter-labs#3006](matter-labs#3006))
([16f2757](matter-labs@16f2757))
* Implement gas relay mode and inclusion data for data attestation
([matter-labs#3070](matter-labs#3070))
([561fc1b](matter-labs@561fc1b))
* **metadata-calculator:** Add debug endpoints for tree API
([matter-labs#3167](matter-labs#3167))
([3815252](matter-labs@3815252))
* **proof-data-handler:** add first processed batch option
([matter-labs#3112](matter-labs#3112))
([1eb69d4](matter-labs@1eb69d4))
* **proof-data-handler:** add tee_proof_generation_timeout_in_secs param
([matter-labs#3128](matter-labs#3128))
([f3724a7](matter-labs@f3724a7))
* **prover:** Add queue metric to report autoscaler view of the queue.
([matter-labs#3206](matter-labs#3206))
([2721396](matter-labs@2721396))
* **prover:** Add sending scale requests for Scaler targets
([matter-labs#3194](matter-labs#3194))
([767c5bc](matter-labs@767c5bc))
* **prover:** Add support for scaling WGs and compressor
([matter-labs#3179](matter-labs#3179))
([c41db9e](matter-labs@c41db9e))
* **vm:** Support EVM emulation in fast VM
([matter-labs#3163](matter-labs#3163))
([9ad1f0d](matter-labs@9ad1f0d))


### Bug Fixes

* **consensus:** better logging of errors
([matter-labs#3170](matter-labs#3170))
([a5028da](matter-labs@a5028da))
* **consensus:** made attestation controller non-critical
([matter-labs#3180](matter-labs#3180))
([6ee9f1f](matter-labs@6ee9f1f))
* **consensus:** payload encoding protected by protocol_version
([matter-labs#3168](matter-labs#3168))
([8089b78](matter-labs@8089b78))
* **da-clients:** add padding to the data within EigenDA blob
([matter-labs#3203](matter-labs#3203))
([8ae06b2](matter-labs@8ae06b2))
* **da-clients:** enable tls-roots feature for tonic
([matter-labs#3201](matter-labs#3201))
([42f177a](matter-labs@42f177a))
* extend allowed storage slots for validation as per EIP-7562
([matter-labs#3166](matter-labs#3166))
([c76da16](matter-labs@c76da16))
* **merkle-tree:** Fix tree truncation
([matter-labs#3178](matter-labs#3178))
([9654097](matter-labs@9654097))
* **tee_prover:** add prometheus pull listener
([matter-labs#3169](matter-labs#3169))
([1ffd22f](matter-labs@1ffd22f))
* update logging in cbt l1 behaviour
([matter-labs#3149](matter-labs#3149))
([d0f61b0](matter-labs@d0f61b0))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: zksync-era-bot <zksync-era-bot@users.noreply.github.com>
)

## What ❔

Move prover_autoscaler config into crate.
Remove protobuf monstrosity.
Create kube client only in Agent.

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

To simplify config management and allow using hashmaps directly in yaml
config.
Also this reduces number of used time crates to 2, reduces build time
and size.

Kube client was created for Scaler too, but was never used.

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.

ref ZKD-1855
## What ❔

If cluster_name isn't provided via flag, get it from GCP. Removes unused
`cluster_name` config option.

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

To reduce possibility of errors in configs.
<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.

ref ZKD-1855
* initial commit

* Add tests

* Add memstore

* Add assert to tests

* Add rest of memstore

* Address pr comments

* Remove to retriable error

* Fix conflicts

* Document memstore

* Fix typo

---------

Co-authored-by: Juan Munoz <juanmunoz890@gmail.com>
blob size limit

---------

Co-authored-by: Gianbelinche <39842759+gianbelinche@users.noreply.github.com>
* initial commit

* Add tests

* Add memstore

* Add assert to tests

* Add rest of memstore

* Add soft confirmations

* Remove print

* Address pr comments

* Remove to retriable error

* Fix conflicts

* Add inclusion data

* Change status query timeout to millis

* Document memstore

* Fix typo

* Fix typo

* Format

---------

Co-authored-by: Juan Munoz <juanmunoz890@gmail.com>
* initial commit

* Add tests

* Add memstore

* Add assert to tests

* Add rest of memstore

* Address pr comments

* Remove to retriable error

* Fix conflicts

* Add verifier

* Fix verifier

* Add path to points to config

* Fix typo

* Fix eigenda env test

* Fix verifier test

* Move eigendaservicemanager to generated

* Remove unneeded imports

* Document verifier

* Modify errors

* Address comments

* Fix conflicts

---------

Co-authored-by: Juan Munoz <juanmunoz890@gmail.com>
* initial commit

* move file location

* fix step
…matter-labs#3214)

## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->
Improve ZK Stack CLI autocomplete CI error message

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.
… in contract verifier (matter-labs#3225)

## What ❔

Supports verifying Solidity contracts deployed with EVM bytecode in the
contract verifier.

## Why ❔

Part of supporting EVM emulation throughout the Era codebase.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.
…abs#3172)

## What ❔

Generalizes Merkle tree recovery so that it supports recovery after the
node has pruned some of its data.

## Why ❔

Improves node UX; allows to recover a tree of a full / non-archive node.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.
…abs#3230)

## What ❔

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->
Add linting target for Rust toolchains.

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.
#329)

* Remove unused custom quorum numbers

* Add serial to tests
juan518munoz and others added 10 commits November 7, 2024 16:32
…utor (matter-labs#3215)

## What ❔

- Adds a new histogram metric <uint type> representing total gas used
per tx
- Adds a new histogram metric <uint type> representing the gas limit per
failed tx

## Why ❔

- The existing metric is a histogram of rates. That's a little difficult
to extract useful throughput numbers out of with the notable exception
of efficiency.
- Exporting this metric to let the consumer do their own: rate/anomoly
calculation on gas/tx, ratio of gas burn for failed tx, etc.

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.
## What ❔
Try to get etup_2\26 key from cache, instead of downloading it from
bucket.

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔
Improve speed for download.
<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.
…abs#3239)

## What ❔
Change renovatebot pr schedule to monthly and grouping changes in one
PR.

<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔
Do not have a lot of PR's and save time to merge them and test.

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [ ] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [ ] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.
…ter-labs#2881)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| trufflesecurity/trufflehog | action | digest | `0c66d30` -> `781157a`
|

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/matter-labs/zksync-era).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC43NC4xIiwidXBkYXRlZEluVmVyIjoiMzkuNy4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…atter-labs#3220)

## What ❔

- Returns just dynamic factory deps (= EVM bytecodes deployed in a
transaction) instead of all deployed bytecodes.
- Tests this functionality in `multivm` and `state_keeper`.

## Why ❔

Returning all factory deps is redundant and makes thing awkward for old
VM versions that don't support EVM emulation.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.
matter-labs#3234)

## What ❔

- Adapts contract verifier APIs to work with EVM bytecodes; adds
corresponding request correctness checks.
- Brushes up the verifier API server in general.

## Why ❔

Part of the efforts to support EVM bytecode verification.

## Checklist

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [x] Tests for the changes have been added / updated.
- [x] Documentation comments have been added / updated.
- [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.
feat(eigen-client-extra-features): fix clippy and add doc
@juanbono juanbono changed the title feat(main): add eigen client features feat(main): add eigen client extra features Nov 8, 2024
yorik and others added 4 commits November 8, 2024 21:07
…hutdown_timeout (matter-labs#3242)

## What ❔

Remove unneeded dependencies.
Add default for graceful_shutdown_timeout.
<!-- What are the changes this PR brings about? -->
<!-- Example: This PR adds a PR template to the repo. -->
<!-- (For bigger PRs adding more context is appreciated) -->

## Why ❔

<!-- Why are these changes done? What goal do they contribute to? What
are the principles behind them? -->
<!-- Example: PR templates ensure PR reviewers, observers, and future
iterators are in context about the evolution of repos. -->

## Checklist

<!-- Check your PR fulfills the following items. -->
<!-- For draft PRs check the boxes as you complete them. -->

- [x] PR title corresponds to the body of PR (we generate changelog
entries from PRs).
- [ ] Tests for the changes have been added / updated.
- [ ] Documentation comments have been added / updated.
- [x] Code has been formatted via `zkstack dev fmt` and `zkstack dev
lint`.

ref ZKD-1855
feat(eigen-client-extra-features): merge main
@juan518munoz
Copy link
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.