Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move long running o1vm test to nightly, run tests for small programs #3010

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

martyall
Copy link
Contributor

@martyall martyall commented Feb 11, 2025

I'm pretty sure I'm using the on.schedule correctly, but I'm unsure how/when it reports to the current PR. The main concern would be that get notified of a failure in the right place

bad news: it takes about the same amount of time as the current job running for 3M instructions (if I understand the current job's --stop-at =3000000 arg). Do I remember that we were going to bump this to use the preimage oracle? In that case maybe its worth it.

I don't think running in parallel will help here as even one program is utilizing all cores on my machine (there are a fair number of parallel commitments being computed)

@martyall martyall force-pushed the martin/o1vm-ci-lightweight branch from e1b82cc to 9c81d15 Compare February 11, 2025 03:27
@martyall martyall force-pushed the martin/o1vm-ci-lightweight branch from 9c81d15 to 32ecf44 Compare February 11, 2025 03:36
Copy link

codecov bot commented Feb 11, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.87%. Comparing base (82bbc48) to head (4888f66).
Report is 10 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3010      +/-   ##
==========================================
+ Coverage   76.83%   76.87%   +0.03%     
==========================================
  Files         262      262              
  Lines       62028    62114      +86     
==========================================
+ Hits        47662    47750      +88     
+ Misses      14366    14364       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@martyall martyall marked this pull request as ready for review February 11, 2025 06:20
@dannywillems
Copy link
Member

Do I remember that we were going to bump this to use the preimage oracle? In that case maybe its worth it.

Yes, and we will need to load a previous state. Discussion on Slack:

re o1vm preimage test in CI: running on my machine gives me a hit of the preimage instruction around 56M and 57M. I would suggest to run the CI from something around ~55M instructions to ~60M instructions.

2025-02-03T12:57:20Z INFO  o1vm::interpreters::mips::witness] processing step=55000000 pc=000023951c insn=00000a1280 ips=91872.79 pages=9167 mem=35.8 MiB name=n/a
[2025-02-03T12:57:21Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:21Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:22Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:23Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:23Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:24Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:24Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:25Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:25Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:26Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:27Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:27Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:28Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:28Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:29Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk
[2025-02-03T12:57:29Z DEBUG o1vm::preimage_oracle] Reading response
[2025-02-03T12:57:29Z DEBUG o1vm::preimage_oracle] Extracting contents
[2025-02-03T12:57:29Z DEBUG o1vm::preimage_oracle] Got preimage of length 32
     dcc3b3317a858eb537c93cc1591da0a1e34dd942610c825e6095fb2b4225a7b0
[2025-02-03T12:57:29Z DEBUG o1vm::interpreters::mips::witness] Preimage has been read entirely, triggering Keccak process
[2025-02-03T12:57:30Z DEBUG o1vm::preimage_oracle] Reading response
[2025-02-03T12:57:30Z DEBUG o1vm::preimage_oracle] Extracting contents
[2025-02-03T12:57:30Z DEBUG o1vm::preimage_oracle] Got preimage of length 32
     7ebd8da4e399db97ac1b854a7a74d6d3bbda59f0dfbb2e0ee024cb4fb80e439e
[2025-02-03T12:57:30Z DEBUG o1vm::interpreters::mips::witness] Preimage has been read entirely, triggering Keccak process
[2025-02-03T12:57:30Z DEBUG o1vm::preimage_oracle] Reading response
[2025-02-03T12:57:30Z DEBUG o1vm::preimage_oracle] Extracting contents
[2025-02-03T12:57:30Z DEBUG o1vm::preimage_oracle] Got preimage of length 32
     cac628b3d4961278fb0461074c1f7df2041887b10511cb6a9099e42a195577e7
[2025-02-03T12:57:30Z DEBUG o1vm::interpreters::mips::witness] Preimage has been read entirely, triggering Keccak process
[2025-02-03T12:57:30Z DEBUG pickles_o1vm] Limit of 65536 reached. We make a proof, verify it (for testing) and start with a new chunk

! -name "oracle*" \
! -name "exit_group" \
! -name "mul" | \
xargs -I {} env RUST_LOG=debug ./o1vm/test-mips-program-prover.sh {} ./srs/test_vesta.srs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed online, I would parallelise it, using an equivalent strategy than in o1js.

Copy link
Member

@dannywillems dannywillems left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refusing for now as it takes > 60m "O1VM MIPS Prover CI / Run o1vm prover on all mips tests (1.74) (pull_request) Successful in 65m" to run the new job. Suggested to parallelize.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants