Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update cargo audit and add to commit #1152

Merged
merged 5 commits into from
Apr 26, 2023
Merged

Update cargo audit and add to commit #1152

merged 5 commits into from
Apr 26, 2023

Conversation

freesig
Copy link
Contributor

@freesig freesig commented Apr 24, 2023

Updates the audit to the latest version and also adds it to PRs

@freesig freesig requested a review from a team April 24, 2023 05:21
@freesig freesig self-assigned this Apr 24, 2023
@freesig
Copy link
Contributor Author

freesig commented Apr 24, 2023 •
edited
Loading

I also removed honeycomb so this can actually pass @Voxelot

@freesig
Copy link
Contributor Author

freesig commented Apr 24, 2023

hmm this workflow seems to just fail randomly

@Voxelot
Copy link
Member

Voxelot commented Apr 24, 2023

Looks like this action isn't compatible with sparse registries yet, it's likely importing an outdated version of cargo:

Caused by:
  usage of sparse registries requires `-Z sparse-registry`

xgreenx
xgreenx reviewed Apr 25, 2023
View reviewed changes
crates/fuel-core/Cargo.toml Outdated Show resolved Hide resolved
crates/fuel-core/src/graphql_api/service.rs
} else {
builder.extension(Tracing)
};
let builder = builder.extension(Tracing);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove tracing at all because we don't use it in the testnet

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still useful to setup the spans used to wrap events logged within the apis

bin/fuel-core/src/cli/run.rs Show resolved Hide resolved
crates/client/src/client.rs Outdated Show resolved Hide resolved
@xgreenx xgreenx mentioned this pull request Apr 25, 2023
Merged
Voxelot
Voxelot previously approved these changes Apr 25, 2023
View reviewed changes
@Voxelot Voxelot dismissed their stale review April 25, 2023 19:40

ci still not passing

@freesig
Copy link
Contributor Author

freesig commented Apr 26, 2023

Looks like this action isn't compatible with sparse registries yet, it's likely importing an outdated version of cargo:

Caused by:
  usage of sparse registries requires `-Z sparse-registry`

How can we fix this? Do we need to fork the action?

@xgreenx
Small clean ups.
6a1c5ac 
Try to use `ubuntu-latest` instead of `buildjet-4vcpu-ubuntu-2204`
@xgreenx xgreenx enabled auto-merge (squash) April 26, 2023 11:01
@xgreenx xgreenx merged commit 2cd6c94 into master Apr 26, 2023
@xgreenx xgreenx deleted the freesig/update-audit branch April 26, 2023 11:02
xgreenx added a commit that referenced this pull request Apr 27, 2023
@xgreenx
Applying optimizations from 0.17.11 release (#1158)
416ce2f 
Closes #1137

The PR duplicates most of the changes from the
#1157 but for the `master`
branch.
It adds two e2e tests, but they fail because of new changes in the
`fuel-vm`(the change to the `PC` register). It is not critical, but
replacing them with the correct scripts later would be nice.
- One test uploads the contract with a huge large state and executes it.
The test takes a lot of time to execute and highlights that we have
performance problems with the state calculation:
#1143
- Another test is more of a base for spamming the testnets. We can put
any hex-encoded transaction and dry run it on the network. It contains a
simple transfer transaction right now.

The change improves the `fuel-core snapshot` command to accept the
arguments, and you can either snap the whole state of the blockchain via
the `everyone` command, or the state only of the contract. In the
follow-up PRs we can snapping of the coins and messages too.

Optimizations:
- Use a lru cache to store the uncompressed data in the DB. The size is
configurable via CLI `max-database-cache-size`.
- Added a `DryRun` execution type that skips the coinbase `Mint`
transaction. In the future, we can skip some non-actual steps too.
- Updated the `PrometheusExtension` to not use the lock.
- Removed usage of the tracing extension in the graph QL because we have
`PrometheusExtension` to gather the statistic about requests.
#1152 removes the honeycomb,
so I didn't duplicate it here.
- Replaced the `tokio::task::spawn_blocking` + `Semaphore` by
`tokio_rayon::spawn_fifo`.
- Replaced the concatenated `column_key` with the array for separate
hash maps per column in the `MemoryStore` and `MemoryTransactionView`.
- Replaced the `Vec<u8>` in the KV API by the `Arc<Vec<u8>>`, because in
most cases, we use it only for deserialization, and the memory database
can continue to store the type. It helps reduce the number of clones.
- Used `BTreeMap` in the `MemoryStore`, because it allows to have a more
optimized `iter_all` method. Without this change, a new e2e test fails
with a timeout.
- Dry run doesn't wait for relayer anymore.
@xgreenx xgreenx mentioned this pull request Apr 27, 2023
Merged
xgreenx added a commit that referenced this pull request Apr 27, 2023
@xgreenx
Release 0.18.0 (#1165)
19f12ea 
The change bumps the version to `0.18.0` and exposes
`sync_max_get_header` and `sync_max_get_txns` in the helm chart.

# Release 0.18.0

## Overview

A new release brings:
- Optimization for the execution based on the performance from beta 3
and on internal benchmarks. Improved metrics gathering.
- Stabilization and better test coverage of the `fuel-vm`. We removed
almost all unsafe code and added test cases for each opcode. Fixed some
edge cases with memory in the `fuel-vm`.
- Fully integrated Merkle trees and filled all malleable fields in the
transactions.
- Added retryable messages, removed redundant fields from it, and
updated the API to support a new commitment schema.

## What's Changed

### Breaking

- All unsafe functions were replaced with safe analog in the
`fuel-crypto` - FuelLabs/fuel-vm#346
- `$hp` holds the address of the last available byte in a heap, while
previously it was `$hp - 1` -
FuelLabs/fuel-vm#377
- Each variant in the `fuel_tx::Input` enum now has its own type -
FuelLabs/fuel-vm#364
- Message nonce is unified and now `Bytes32` everywhere -
FuelLabs/fuel-vm#394
- Removed the `message_id` field from all places -
FuelLabs/fuel-vm#397,
FuelLabs/fuel-vm#373,
- Unified the block height in all places with the introduction of a new
`BlockHeigh` - FuelLabs/fuel-vm#410
- Make SMO instruction take data ptr as an argument -
FuelLabs/fuel-vm#404
- Now the chain id affects the signature and predicate's owner and
should be passed into the `sign` function -
FuelLabs/fuel-vm#406
- Updated the `produce_blocks` endpoint to accept the start time and the
number of blocks. All new blocks will use the previous timestamp as a
base - #1059
- The `fuel-core` stores only unspent coins and messages, so all API
that previously returned spent coins is affected - Prune owned coin idx
when inputs are spent by @Voxelot in
#1055
- The message proof API has been changed to be compatible with a new
version - #1071
- The `fuel-core` now has retryable messages and coin messages.
Retryable messages can only be consumed during successful transaction
execution. The coin message acts as common coins. `resouces_to_spend`
API was replaced with `coins_to_spend` that returns a new `CoinType`
type. - #1067

## All changes
* adding fuel-core service monitor to helm chart by @rfuelsh in
#1037
* Adding specific app selector by @rfuelsh in
#1039
* use sticky sessions for GQL requests to sentries by @Voxelot in
#1042
* Add ingress service name to helm chart by @rfuelsh in
#1043
* Added test to verify the iteration over owned transactions by @xgreenx
in #1041
* Change sentry lb to use clusterIP by @Voxelot in
#1045
* Fixed Tempfile dependency by @ControlCplusControlV in
#1048
* Write contract code in raw by @freesig in
#994
* attempt to use buildjet runners by @Voxelot in
#1017
* Set tx pointer during block production by @Voxelot in
#1054
* Used `BASE_AMOUNT` for test with bob to pay for fee by @xgreenx in
#1057
* Prepare GraphQL Crate for extraction by @ControlCplusControlV in
#1012
* Support large contract deployments over p2p by @Voxelot in
#1062
* fix yaml and add linting job by @Voxelot in
#1063
* Actualized the ABI for message to be compatible with last version of
the Solidity contracts. by @xgreenx in
#1060
* feat: Contract state and assets merkle data storage by @bvrooman in
#1008
* Take into account the previous block timestamp during block production
by @xgreenx in #1059
* Prune owned coin idx when inputs are spent by @Voxelot in
#1055
* Retrayable messages fuel-core part by @xgreenx in
#1067
* chore: Additional Tests for Contract States and Balances by @bvrooman
in #1073
* Rust 1.68.1 & Sparse Registry by @Voxelot in
#1077
* Nginx tuneup by @Voxelot in
#1080
* chore: Update balance and state Merkleization by @bvrooman in
#1084
* Support sticky session in the client by @xgreenx in
#1079
* Added e2e test to run 1000 `dry_run` by @xgreenx in
#1091
* Use docker.io auth credentials to overcome rate limiting by @Voxelot
in #1092
* honeycomb tracing subscription by @Voxelot in
#1083
* Update withdrawal proof API to support proving from a block header
committed to L1 by @xgreenx in
#1071
* Upstream v0.17.8 by @Voxelot in
#1102
* use chainid for transactions and predicates by @Voxelot in
#1107
* fix the dry-run e2e test to actually perform dry runs by @Voxelot in
#1111
* peer reputation with separate app & gossipsub score by @leviathanbeak
in #1028
* Adding sentry data synchronization by @rfuelsh in
#1115
* Capture metrics for graphql api by @Voxelot in
#1113
* Update cargo audit and add to commit by @freesig in
#1152
* add task params by @leviathanbeak in
#1159
* Took into account that block creation could take some time by @xgreenx
in #1162
* Applying optimizations from `0.17.11` release by @xgreenx in
#1158


**Full Changelog**:
v0.17.3...v0.18.0
crypto523 pushed a commit to crypto523/fuel-core that referenced this pull request Oct 7, 2024
Applying optimizations from 0.17.11 release (#1158)
c6039c9 
Closes FuelLabs/fuel-core#1137

The PR duplicates most of the changes from the
FuelLabs/fuel-core#1157 but for the `master`
branch.
It adds two e2e tests, but they fail because of new changes in the
`fuel-vm`(the change to the `PC` register). It is not critical, but
replacing them with the correct scripts later would be nice.
- One test uploads the contract with a huge large state and executes it.
The test takes a lot of time to execute and highlights that we have
performance problems with the state calculation:
FuelLabs/fuel-core#1143
- Another test is more of a base for spamming the testnets. We can put
any hex-encoded transaction and dry run it on the network. It contains a
simple transfer transaction right now.

The change improves the `fuel-core snapshot` command to accept the
arguments, and you can either snap the whole state of the blockchain via
the `everyone` command, or the state only of the contract. In the
follow-up PRs we can snapping of the coins and messages too.

Optimizations:
- Use a lru cache to store the uncompressed data in the DB. The size is
configurable via CLI `max-database-cache-size`.
- Added a `DryRun` execution type that skips the coinbase `Mint`
transaction. In the future, we can skip some non-actual steps too.
- Updated the `PrometheusExtension` to not use the lock.
- Removed usage of the tracing extension in the graph QL because we have
`PrometheusExtension` to gather the statistic about requests.
FuelLabs/fuel-core#1152 removes the honeycomb,
so I didn't duplicate it here.
- Replaced the `tokio::task::spawn_blocking` + `Semaphore` by
`tokio_rayon::spawn_fifo`.
- Replaced the concatenated `column_key` with the array for separate
hash maps per column in the `MemoryStore` and `MemoryTransactionView`.
- Replaced the `Vec<u8>` in the KV API by the `Arc<Vec<u8>>`, because in
most cases, we use it only for deserialization, and the memory database
can continue to store the type. It helps reduce the number of clones.
- Used `BTreeMap` in the `MemoryStore`, because it allows to have a more
optimized `iter_all` method. Without this change, a new e2e test fails
with a timeout.
- Dry run doesn't wait for relayer anymore.
crypto523 pushed a commit to crypto523/fuel-core that referenced this pull request Oct 7, 2024
Release 0.18.0 (#1165)
2d2a6a3 
The change bumps the version to `0.18.0` and exposes
`sync_max_get_header` and `sync_max_get_txns` in the helm chart.

# Release 0.18.0

## Overview

A new release brings:
- Optimization for the execution based on the performance from beta 3
and on internal benchmarks. Improved metrics gathering.
- Stabilization and better test coverage of the `fuel-vm`. We removed
almost all unsafe code and added test cases for each opcode. Fixed some
edge cases with memory in the `fuel-vm`.
- Fully integrated Merkle trees and filled all malleable fields in the
transactions.
- Added retryable messages, removed redundant fields from it, and
updated the API to support a new commitment schema.

## What's Changed

### Breaking

- All unsafe functions were replaced with safe analog in the
`fuel-crypto` - FuelLabs/fuel-vm#346
- `$hp` holds the address of the last available byte in a heap, while
previously it was `$hp - 1` -
FuelLabs/fuel-vm#377
- Each variant in the `fuel_tx::Input` enum now has its own type -
FuelLabs/fuel-vm#364
- Message nonce is unified and now `Bytes32` everywhere -
FuelLabs/fuel-vm#394
- Removed the `message_id` field from all places -
FuelLabs/fuel-vm#397,
FuelLabs/fuel-vm#373,
- Unified the block height in all places with the introduction of a new
`BlockHeigh` - FuelLabs/fuel-vm#410
- Make SMO instruction take data ptr as an argument -
FuelLabs/fuel-vm#404
- Now the chain id affects the signature and predicate's owner and
should be passed into the `sign` function -
FuelLabs/fuel-vm#406
- Updated the `produce_blocks` endpoint to accept the start time and the
number of blocks. All new blocks will use the previous timestamp as a
base - FuelLabs/fuel-core#1059
- The `fuel-core` stores only unspent coins and messages, so all API
that previously returned spent coins is affected - Prune owned coin idx
when inputs are spent by @Voxelot in
FuelLabs/fuel-core#1055
- The message proof API has been changed to be compatible with a new
version - FuelLabs/fuel-core#1071
- The `fuel-core` now has retryable messages and coin messages.
Retryable messages can only be consumed during successful transaction
execution. The coin message acts as common coins. `resouces_to_spend`
API was replaced with `coins_to_spend` that returns a new `CoinType`
type. - FuelLabs/fuel-core#1067

## All changes
* adding fuel-core service monitor to helm chart by @rfuelsh in
FuelLabs/fuel-core#1037
* Adding specific app selector by @rfuelsh in
FuelLabs/fuel-core#1039
* use sticky sessions for GQL requests to sentries by @Voxelot in
FuelLabs/fuel-core#1042
* Add ingress service name to helm chart by @rfuelsh in
FuelLabs/fuel-core#1043
* Added test to verify the iteration over owned transactions by @xgreenx
in FuelLabs/fuel-core#1041
* Change sentry lb to use clusterIP by @Voxelot in
FuelLabs/fuel-core#1045
* Fixed Tempfile dependency by @ControlCplusControlV in
FuelLabs/fuel-core#1048
* Write contract code in raw by @freesig in
FuelLabs/fuel-core#994
* attempt to use buildjet runners by @Voxelot in
FuelLabs/fuel-core#1017
* Set tx pointer during block production by @Voxelot in
FuelLabs/fuel-core#1054
* Used `BASE_AMOUNT` for test with bob to pay for fee by @xgreenx in
FuelLabs/fuel-core#1057
* Prepare GraphQL Crate for extraction by @ControlCplusControlV in
FuelLabs/fuel-core#1012
* Support large contract deployments over p2p by @Voxelot in
FuelLabs/fuel-core#1062
* fix yaml and add linting job by @Voxelot in
FuelLabs/fuel-core#1063
* Actualized the ABI for message to be compatible with last version of
the Solidity contracts. by @xgreenx in
FuelLabs/fuel-core#1060
* feat: Contract state and assets merkle data storage by @bvrooman in
FuelLabs/fuel-core#1008
* Take into account the previous block timestamp during block production
by @xgreenx in FuelLabs/fuel-core#1059
* Prune owned coin idx when inputs are spent by @Voxelot in
FuelLabs/fuel-core#1055
* Retrayable messages fuel-core part by @xgreenx in
FuelLabs/fuel-core#1067
* chore: Additional Tests for Contract States and Balances by @bvrooman
in FuelLabs/fuel-core#1073
* Rust 1.68.1 & Sparse Registry by @Voxelot in
FuelLabs/fuel-core#1077
* Nginx tuneup by @Voxelot in
FuelLabs/fuel-core#1080
* chore: Update balance and state Merkleization by @bvrooman in
FuelLabs/fuel-core#1084
* Support sticky session in the client by @xgreenx in
FuelLabs/fuel-core#1079
* Added e2e test to run 1000 `dry_run` by @xgreenx in
FuelLabs/fuel-core#1091
* Use docker.io auth credentials to overcome rate limiting by @Voxelot
in FuelLabs/fuel-core#1092
* honeycomb tracing subscription by @Voxelot in
FuelLabs/fuel-core#1083
* Update withdrawal proof API to support proving from a block header
committed to L1 by @xgreenx in
FuelLabs/fuel-core#1071
* Upstream v0.17.8 by @Voxelot in
FuelLabs/fuel-core#1102
* use chainid for transactions and predicates by @Voxelot in
FuelLabs/fuel-core#1107
* fix the dry-run e2e test to actually perform dry runs by @Voxelot in
FuelLabs/fuel-core#1111
* peer reputation with separate app & gossipsub score by @leviathanbeak
in FuelLabs/fuel-core#1028
* Adding sentry data synchronization by @rfuelsh in
FuelLabs/fuel-core#1115
* Capture metrics for graphql api by @Voxelot in
FuelLabs/fuel-core#1113
* Update cargo audit and add to commit by @freesig in
FuelLabs/fuel-core#1152
* add task params by @leviathanbeak in
FuelLabs/fuel-core#1159
* Took into account that block creation could take some time by @xgreenx
in FuelLabs/fuel-core#1162
* Applying optimizations from `0.17.11` release by @xgreenx in
FuelLabs/fuel-core#1158


**Full Changelog**:
FuelLabs/fuel-core@v0.17.3...v0.18.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Voxelot Voxelot Voxelot left review comments

@xgreenx xgreenx xgreenx approved these changes

Assignees

@freesig freesig

Labels
audit-blocker security
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@freesig @Voxelot @xgreenx