Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update from Main #3831

Merged
merged 135 commits into from
Apr 17, 2023
Merged

Update from Main #3831

merged 135 commits into from
Apr 17, 2023

Conversation

dr-chain
Copy link
Contributor

Update from main

AurelienFT and others added 30 commits March 1, 2023 10:25
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
* initial fix

* Do not set ipv6 flag if ipv4 only

* fmt

* Added documentation and comments
* fix(vesting) : fix init slot with passed timestamp #3654
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
* Initial code for versioning & versioning factory models

* Update VersioningStateHistory && add VersioningStore & VersioningStoreRaw

* VersioningStoreRaw::merge_with -> update_with

* Implement Factory methods

* Clippy pass

* Renaming + add component version

* Move into massa-versioning-worker && massa-versioning-exports

* Improve doc in versioning.rs

* Avoid storing too much things in history for MipStateHistory + use network_version & component_version

* Best -> Latest & update crate documentation

* MipState -> ComponentState && MipStateHistory -> MipState

* Cargo clipp fixes

* cargo fmt for tests/mod.rs file

* Remove unused import in test_helpers/versioning_helpers.rs

* Remove some warnings about dead code in unit tests

* Fix machine crate failure in doctests

---------

Co-authored-by: sydhds <sd@massa.net>
Add colors on the get_operations command
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
Signed-off-by: Litchi Pi <litchi.pi@proton.me>
sydhds and others added 29 commits April 4, 2023 09:40
Co-authored-by: sydhds <sd@massa.net>
Co-authored-by: modship <lu@massa.net>
* Create massa_sc_module_cache

* Change ModuleCache documentation and add HDCache placeholder functions

* Create LRUCache and implement load_module

* Impl save_module_from_bytecode and correct mistakes

* Separate init costs handling and update save_module_from_bytecode

* Add remove_module

* Use latest runtime changes, fix behaviours and add comments

* Add HDCache

Signed-off-by: Jean-François Morcillo <jfm@massa.net>

* Fix internal api

Signed-off-by: Jean-François Morcillo <jfm@massa.net>

* Upate init_costs handling and add ModuleInfo

* Fix missing occurences of ModuleInfo

* temporary cache usage update on execution, not expected behaviour

* fixup! Add HDCache

* REMOVE ME change dep for massa-sc-runtime

Signed-off-by: Jean-François Morcillo <jfm@massa.net>

* update load_module, use set_init_cost, add update_module

* Minor changes

* fix get()

Signed-off-by: Jean-François Morcillo <jfm@massa.net>

* Revert "REMOVE ME change dep for massa-sc-runtime"

This reverts commit 76276e3.

* Make insert en remove atomic, fix internal interface, make tests pass

/!\ tests are ok if run 1 by 1 but fail if run in parallel

Signed-off-by: Jean-François Morcillo <jfm@massa.net>

* Add upcomming ModuleInfo type

* Limit number of entries in the db, adjust API

Signed-off-by: Jean-François Morcillo <jfm@massa.net>

* Select entries to remove randomly

Signed-off-by: Jean-François Morcillo <jfm@massa.net>

* ModuleInfo NOM Ser & Deser

* Wip transition to ModuleInfoBis

Signed-off-by: Jean-François Morcillo <jfm@massa.net>

* No SP cache implementation

* Use latest impl in execution

* Intermediate commit for hd cache update

* Cache update on settle slot

* max block gas as first input limit for compilation

* Update keys format in hd cache, metadata value serializers

* Rename metadata fields

* Rename controller functions

* Re impl set_init_cost and remove cf usage

* Impl get

* Update save_module, implement set_invalid, change lru error handling

* Update fix impl details and start changing error handling

* Update error handling

* Rename cache module

* Update hd cache get behaviour

* Update load_module behaviour

* test update

* Hd cache tests update

* Add test_missing_module

* Fix hd cache get behaviour

* Fix invalid module deser error

* Cache config

* Fill basic cache config

* tmp dir on test cache creation

* Complete config setup

* Update ExecutionConfig test default value & disable test_hash_sha256

* Fix on ledger change cache update error

* Update todos

* Minor clippy fix

* Update controller error handling, hd cache snip behaviour, massa-node settings input

* Doc

* Compilation costs and doc

* Fix hd cache tests

* Debit compilation costs

* Debit cost on tmp module execution

* Test compilation costs payment

* Update compilation costs to associate it with gas

* Revert previous gas costs tests and add not_enough_compilation_gas

* Fix clippy warnings

* Fix node settings

* Cache controller logs

* Remove unecessary module cloning

* Use latest runtime changes in cache controller

* Fix parent branch failing test_hash_sha256

* Review update + Revert parent branch test fixes

* Save init costs on VM execution error and update error handling

* Improve hd cache snip doc

* Add context to the VM related execution errors

* Fix not_enough_compilation_gas test

* Fix vesting tests

* Logs and safety check

* Update runtime version

---------

Signed-off-by: Jean-François Morcillo <jfm@massa.net>
Co-authored-by: Jean-François Morcillo <jfm@massa.net>
* Initial commit for network restart: Archive read / write

Needs testing

* Skeleton of the snapshot creation workflow

* Removed archiving tests

* fmt

* cleaned a bit the FinalState init

* Cleaned (check / clippy)

* Added Pos deserialization

* Added final state serialization to file

* Refactoring + create final_state dir if it does not exist

* fixed path issue

* Reverted some commits of local config

* Don't reset ledger if from snapshot (will be changed for testnet21)

Indeed, we will always keep the ledger from now on.

* Compute and check hash of recovered final_state

* Don't replace old ledger

* Changed timestamps to account for LAST_START_PERIOD

But no finalized blocks are made anymore, I'll have to check the concensus rules...

* fmt

* Fixed some final_cursor issues

create_snapshot and --from_snapshot should work well in sandbox mode!

* clippy allow lint in case the constant is not 0

* Changed LAST_START_PERIOD from const to static ref

+ Ledger rollback based on genesis_timestamp

* Fixed cycle_history issue and block_header deserialize assumption

* fmt

* Fixed hash computation in --from_snapshot

* cleaned the code

* Rethought the cycle logic when LAST_STAR_PERIOD is not 0.

* Keep previous roll_counts

* fmt

* clippy

* Fixed initial_rolls when snapshotting

* Added feed_cycle_state_hash in --from_snapshot

Don't know if I should do it or not :/

* fmt

* Set up PoS_final_state after bootstrap

* fmt

* Added debug info

* Fix Mutex lock issue

* change config for tests

* Added bootstrap of initial pos_state info

- initial_cycle
- initial_rolls
- initial_ledger_hash

* Change desynchronisation checks

* Added cycle_history debug info

* Boostrapable last_start_period value

* Final_state in ledger

* Fixed final state in ledger

The dump / commit logic is probably false right now, need to investigate the best way to handle it

* doctest pass

* Fix bootstrap message deser

* disable desync detection + bootstrap fix

* Fixed deployment (lazy_static parsing fail) + fmt

* logs for cycle info changes

* Fixed execution cycle lookback

* Tried to fix the draws

* Updated deferred credit logic

* Fixed deferred credit hash for snapshot + removed info!()

* Make test pass + clean info!() logs

* Removed final_state_path setting, no longer used

* Added documentation, restored periods_per_cycle at 128

* Refactoring + Only bootstrap the initial_state once

* Update massa-final-state/src/final_state.rs

Co-authored-by: Ben <benphawke@gmail.com>

* fmt

* Cleaned the FinalState::from_snapshot() method

Created Ser/Deser helpers for FinalStateRaw.

* Better handle final_state in RocksDB

* Added a TODO for get_pinned

* Use range() on the BTreeMap to better filter the slot range.

* Fixed a regretion in the previous commit

* Doc comments lints

* Doc comment for final_state_data in ledger

* cleaned final_state_raw_deserializer args

* Used get_pinned_cf for final_state_hash

* fmt + _args => args

* Changed final_state dispatch in main

* Interpolate the cycles (untested)

* Small fixes and lints

* Delete intiial_state logic (bootstrap, deser, ...) + iniitial_cycle for pos_state

* fmt

* clippy fix

* update snapshot logic without get_final_state

* Set period_per_cycle to 10 for tests

* Fix feed_cycle_hash

* fmt

* Feed selector during downtime interpolation

* feed_selector and compute draws logic added

* clippy fix

* Don't dump the final_state on finalize()

* Added comments for periods_per_cycle

* small refactoring and docs update for pos_final_state

* Make CI pass

Cf #3742

* Small refactor of final_state

* Add TODO for commented imports

* Refactor final_state downtime interpolation

* Removed an unneeded rocks_db call

* Included (x-1) > Excluded (x) in bounds

* fmt

* Updated deferred_credits logic, TODO: TESTS

Inspired from #3740

* Small cleanup / renames

* Updated roll_sell test to test the new deferred_credits impl

* Cleanup after changes: no more last_start_period needed in pos_final_state

* TODO: CHANGE BACK CONST

* Fixed block_factory next_slot if before last_start_period

* Update final_state.rs

* Changed rng_seed push to avoid loops

* info > debug for all denuncuations

* Revert "info > debug for all denuncuations"

This reverts commit 2b28b33.

* Make PR Ready!

* Fix merge with GRPC

* Improved some docs

* Fix execution test after merge

* New pass over documentation

* Final doc improvements

* Apply suggestions from code review

Co-authored-by: Ben <benphawke@gmail.com>

* Applied most of the suggestions in review

* fmt

* fixed stats feature flags

* fmt

---------

Co-authored-by: Ben <benphawke@gmail.com>
* Add a downtime simulator in consensus

1 hour after genesis, for a 1 hour duration.

* Add comment for last_start_period calculation

* Added check at start of main, and use Timestamps
* Panics if we are before genesis timestamp

Used to prevent nodes to desync in case we keep a different ledger than they have.

* Deactivate the check if we are in sandbox
* Add stop command consensus

* Format

* Add comment to clarify why we send stop command
* Add mip store bootstrap

* Now update main MIP store with what is received from bootstrap

* Cargo clippy fixes

* Fix doc test

---------

Co-authored-by: sydhds <sd@massa.net>
* clean the other branch

* fmt

* TODO: roll  periods_per_cycle back

* info on cycle_history

* fixed hash computation for cycle_hash

* fixed rng_seed length for 1st one

* Final fix :)

* Revert logs + periods_per_cycle + fmt 🎉

* Update downtime timestamps

10AM > 11AM on 6th of april
Co-authored-by: AurelienFT <aurelien.foucault@epitech.eu>
* Rework denunciation factory block header denunciation test

* Fix doc test

* Add denunciation factory unit test for denunciation factory

* Move crossbeam-channel behind feature testing

* Remove unused dependency in unit test

---------

Co-authored-by: sydhds <sydhds@gmail.com>
Co-authored-by: sydhds <sydhds@gmail.com>
3620: Testnet 21 r=AurelienFT a=AurelienFT



Co-authored-by: AurelienFT <aurelien.foucault@epitech.eu>
Co-authored-by: AurelienFT <32803821+AurelienFT@users.noreply.github.com>
Co-authored-by: Sydhds <sylvain.delhomme@gmail.com>
Co-authored-by: Leo-Besancon <lb@massa.net>
Co-authored-by: Moncef AOUDIA <22281426+aoudiamoncef@users.noreply.github.com>
Co-authored-by: Litchi Pi (Tim) <litchi.pi@proton.me>
Co-authored-by: Litchi Pi <litchi.pi@proton.me>
Co-authored-by: Ben <benphawke@gmail.com>
Co-authored-by: Eitu33 <89928840+Eitu33@users.noreply.github.com>
Co-authored-by: Massembler <af@massa.net>
Bumps [h2](https://github.com/hyperium/h2) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/hyperium/h2/releases)
- [Changelog](https://github.com/hyperium/h2/blob/master/CHANGELOG.md)
- [Commits](hyperium/h2@v0.3.16...v0.3.17)

---
updated-dependencies:
- dependency-name: h2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
3825: Bump h2 from 0.3.16 to 0.3.17 r=AurelienFT a=dependabot[bot]

Bumps [h2](https://github.com/hyperium/h2) from 0.3.16 to 0.3.17.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/hyperium/h2/releases">h2's releases</a>.</em></p>
<blockquote>
<h2>v0.3.17</h2>
<h2>What's Changed</h2>
<ul>
<li>Add <code>Error::is_library()</code> method to check if the originated inside <code>h2</code>.</li>
<li>Add <code>max_pending_accept_reset_streams(usize)</code> option to client and server
builders.</li>
<li>Fix theoretical memory growth when receiving too many HEADERS and then
RST_STREAM frames faster than an application can accept them off the queue.
(CVE-2023-26964)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/hyperium/h2/blob/master/CHANGELOG.md">h2's changelog</a>.</em></p>
<blockquote>
<h1>0.3.17 (April 13, 2023)</h1>
<ul>
<li>Add <code>Error::is_library()</code> method to check if the originated inside <code>h2</code>.</li>
<li>Add <code>max_pending_accept_reset_streams(usize)</code> option to client and server
builders.</li>
<li>Fix theoretical memory growth when receiving too many HEADERS and then
RST_STREAM frames faster than an application can accept them off the queue.
(CVE-2023-26964)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/hyperium/h2/commit/af4bcacf6d3770e9e3dc10fdc631fc8c0bdd472b"><code>af4bcac</code></a> v0.3.17</li>
<li><a href="https://github.com/hyperium/h2/commit/d3f37e9fbad6608ba74419c355eb1892bd55d977"><code>d3f37e9</code></a> feat: add <code>max_pending_accept_reset_streams(n)</code> options</li>
<li><a href="https://github.com/hyperium/h2/commit/5bc8e72e5fcbd8ae2d3d9bc78a1c0ef0040bcc39"><code>5bc8e72</code></a> fix: limit the amount of pending-accept reset streams</li>
<li><a href="https://github.com/hyperium/h2/commit/8088ca658d90a3874fb6b136b85776424265295b"><code>8088ca6</code></a> feat: add Error::is_library method</li>
<li><a href="https://github.com/hyperium/h2/commit/481c31d5283bf32b90c83388c037494548934971"><code>481c31d</code></a> chore: Use Cargo metadata for the MSRV build job</li>
<li><a href="https://github.com/hyperium/h2/commit/d3d50ef8123f0a1b6d16faa2d9edc01418da7c00"><code>d3d50ef</code></a> chore: Replace unmaintained/outdated GitHub Actions</li>
<li><a href="https://github.com/hyperium/h2/commit/45b9bccdfcb26cfe9907123a1e975a22eb84c44f"><code>45b9bcc</code></a> chore: set rust-version in Cargo.toml (<a href="https://redirect.github.com/hyperium/h2/issues/664">#664</a>)</li>
<li>See full diff in <a href="https://github.com/hyperium/h2/compare/v0.3.16...v0.3.17">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=h2&package-manager=cargo&previous-version=0.3.16&new-version=0.3.17)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/massalabs/massa/network/alerts).

</details>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@dr-chain dr-chain merged commit c730b4e into dr-chain-patch-1 Apr 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.