Skip to content

feat: cache simnet remote metadata #1732

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 3, 2025
Merged

Conversation

hugoclrd
Copy link
Collaborator

@hugoclrd hugoclrd commented Apr 1, 2025

Description

In the context of mainnet execution simulation (MXS), the simnet fetches a lot of data. This data is immutable and can easily be cached.
This PR only stores the metadata, making the MXS already way faster.
In future work, we could also cache all of the marf values or add an option to do so.

The FileSystem handling strategy, especially for wasm, is different from other approaches existing in the repo. But it's simpler and should be the proferred strategy from now on

  • until now, we were passing a WASMFileAccessor from JS to Rust to handle fs operation. It was trciky to get this down to the datastore, and we needed the FS to be handle synchronously.
  • we are instead directly:
    • directly accessing node fs in the context of ndoe.js
    • or setting up a globally avaialble vfs() function in the context of web browser.

See this code:

#[cfg(all(target_arch = "wasm32", feature = "web"))]
mod browser;
#[cfg(all(target_arch = "wasm32", feature = "web"))]
pub use browser::*;
#[cfg(all(target_arch = "wasm32", not(feature = "web")))]
mod node;
#[cfg(all(target_arch = "wasm32", not(feature = "web")))]
pub use node::*;

@hugoclrd hugoclrd force-pushed the feat/mainnet-simulation-enhancements branch from 8059aaa to f1f4639 Compare April 1, 2025 14:58
Copy link

codecov bot commented Apr 1, 2025

Codecov Report

Attention: Patch coverage is 95.94595% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
components/clarinet-files/src/lib.rs 80.00% 1 Missing ⚠️
components/clarity-repl/src/repl/datastore.rs 97.22% 1 Missing ⚠️
...nts/clarity-repl/src/repl/remote_data/fs/native.rs 92.30% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@hugoclrd hugoclrd requested a review from tippenein April 1, 2025 16:14
@hugoclrd hugoclrd force-pushed the feat/mainnet-simulation-enhancements branch 2 times, most recently from 8146575 to 9d143ec Compare April 2, 2025 15:12
tippenein
tippenein previously approved these changes Apr 2, 2025
Copy link
Collaborator

@tippenein tippenein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! This seems like it’ll be useful for other optimizations for sure

@hugoclrd hugoclrd force-pushed the feat/mainnet-simulation-enhancements branch from 29a2293 to 2eed17d Compare April 3, 2025 14:13
@hugoclrd hugoclrd enabled auto-merge (squash) April 3, 2025 14:47
@hugoclrd hugoclrd merged commit 6758937 into main Apr 3, 2025
22 checks passed
@hugoclrd hugoclrd deleted the feat/mainnet-simulation-enhancements branch April 3, 2025 14:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants