Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add database analyze-contract-sizes #11249

Merged
merged 4 commits into from
May 14, 2024
Merged

Add database analyze-contract-sizes #11249

merged 4 commits into from
May 14, 2024

Conversation

jancionear
Copy link
Contributor

@jancionear jancionear commented May 7, 2024 •
edited
Loading

Add a command which looks at the current state and finds accounts with the biggest contract sizes.

Example usage:

$ ./neard database analyze-contract-sizes --topn 10
...
Analyzed 33165 accounts
Accounts per shard:
s0.v3: 6443
s1.v3: 1
s2.v3: 7916
s3.v3: 3643
s4.v3: 11208
s5.v3: 3954

Top 10 accounts by contract size:
3.9 MB: nearpay-portals.near
3.8 MB: moodev.near
2.8 MB: v2_1_0.perp.spin-fi.near
2.8 MB: v2_0_2.perp.spin-fi.near
2.8 MB: v2.perp.spin-fi.near
2.7 MB: switchboard-v2.near
2.6 MB: nft.contented.near
2.5 MB: spot.spin-fi.near
2.5 MB: nftstaking.jumpfinance.near
2.5 MB: exchange.slowisfast.near

@jancionear
Add database analyze-contract-sizes
4ff9075 
Add a command which looks at the current state
and finds accounts with biggest contract sizes.

Example usage:

```
$ ./neard database analyze-contract-sizes --topn 10
...
Analyzed 33165 accounts
Accounts per shard:
s0.v3: 6443
s1.v3: 1
s2.v3: 7916
s3.v3: 3643
s4.v3: 11208
s5.v3: 3954

Top 10 accounts by size:
3.9 MB: nearpay-portals.near
3.8 MB: moodev.near
2.8 MB: v2_1_0.perp.spin-fi.near
2.8 MB: v2_0_2.perp.spin-fi.near
2.8 MB: v2.perp.spin-fi.near
2.7 MB: switchboard-v2.near
2.6 MB: nft.contented.near
2.5 MB: spot.spin-fi.near
2.5 MB: nftstaking.jumpfinance.near
2.5 MB: exchange.slowisfast.near
```
@jancionear jancionear requested a review from Longarithm May 7, 2024 12:32
@jancionear jancionear requested a review from a team as a code owner May 7, 2024 12:32
@codecov Codecov
Copy link

codecov bot commented May 7, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 2.29885% with 85 lines in your changes are missing coverage. Please review.

Project coverage is 71.12%. Comparing base (7ec80b3) to head (95b86ad).
Report is 16 commits behind head on master.

Files Patch % Lines
tools/database/src/analyze_contract_sizes.rs 2.32% 83 Missing and 1 partial ⚠️
tools/database/src/commands.rs 0.00% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master   #11249      +/-   ##
==========================================
+ Coverage   64.93%   71.12%   +6.19%     
==========================================
  Files         771      785      +14     
  Lines      153218   155330    +2112     
  Branches   153218   155330    +2112     
==========================================
+ Hits        99486   110485   +10999     
+ Misses      49494    40098    -9396     
- Partials     4238     4747     +509
Flag Coverage Δ
backward-compatibility 0.24% <0.00%> (?)
db-migration 0.24% <0.00%> (?)
genesis-check 1.41% <2.29%> (?)
integration-tests 36.90% <0.00%> (?)
linux 69.12% <2.29%> (+4.19%) ⬆️
linux-nightly 70.58% <2.29%> (?)
macos 53.02% <2.29%> (?)
pytests 1.63% <2.29%> (?)
sanity-checks 1.42% <2.29%> (?)
unittests 65.86% <2.29%> (+0.93%) ⬆️
upgradability 0.29% <2.29%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Longarithm
Longarithm approved these changes May 8, 2024
View reviewed changes
tools/database/src/analyze_contract_sizes.rs Outdated
Comment on lines 90 to 92
if last_block.header().chunk_mask().contains(&false) {
panic!("Last block doesn't have all chunks, please try again later!");
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hehe :) Consider taking ChunkExtras for last block - it will have all state roots. Alternatively we can iterate back in history to find block with all chunks, but we ever have 100 shards, that can become unstable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh and ChunkExtra will exist even for missing chunks? That's nice, changed it to use ChunkExtra then.

tools/database/src/analyze_contract_sizes.rs Outdated
Comment on lines 102 to 105
let trie = Trie::new(trie_storage, state_root, None);

let mut iterator = trie.iter().unwrap();
iterator.seek_prefix(&[col::CONTRACT_CODE]).unwrap();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively we can look at flat storage head and read all data from FlatState. But if this is quick already / we already have all the results, proceed with Trie.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It takes under 30s to analyze the current mainnet state on dev-release, so I think it's fast enough.

@jancionear jancionear added this pull request to the merge queue May 14, 2024
Merged via the queue into near:master with commit 8441c93 May 14, 2024
29 checks passed
@jancionear jancionear deleted the analyze-contract-sizes branch May 14, 2024 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Longarithm Longarithm Longarithm approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jancionear @Longarithm