Feat: Native Poseidon2 chip

[TlatoaniHJ]

#1133, INT-2947

Adds a chip to complete all of verify_batch in a single instruction. This is meant to replace the current NativePoseidon2Chip. Associated with this, this PR adds a new OpCode (currently called VERIFY_BATCH) and two new AsmInstruction and DslIr variants, one each for the two cases where the opened values are Felts and Exts.

The instruction has 7 operands: dim, opened, opened_len, sibling, index, commit, opened_value_size_inv. dim, opened, sibling, index, commit are the locations of the pointers for the respective arrays; opened_len is the location of the length of the opened values array, and opened_value_size_inv is 1 when the opened values are Felts and 1/4 when the opened values are Exts.

The chip itself works by having two types of rows: top level, and inside row. Top level rows are responsible for merging in the Merkle tree, while inside row rows are responsible for performing the rolling hash of the concatenated rows. The last top level row is responsible for the execution interaction. There are two types of top level rows: incorporate row, which are responsible for merging in the hash of the concatenated rows, and incorporate sibling, which are responsible for merging in the siblings. Incorporate row rows send an interaction to the last inside row row in each segment of inside row rows.

---

Copilot:
This pull request focuses on removing the Poseidon2 opcode and related code, and introducing a new VERIFY_BATCH opcode. Additionally, it includes some refactoring and the addition of new methods. Here are the most important changes:

Opcode Changes:

• Changed opcode_offset for PublishOpcode from 0x120 to 0x020 in crates/toolchain/instructions/src/lib.rs. Removed Poseidon2Opcode and related code.
• Added VERIFY_BATCH opcode tailored for optimized verification in docs/specs/ISA.md.

Code Removal:

• Removed all references to Poseidon2Opcode in crates/vm/tests/integration_test.rs and related test functions. [1] [2] [3] [4]

Refactoring:

• Refactored the NativePoseidon2Cols struct and added new structs TopLevelSpecificCols, InsideRowSpecificCols, and SimplePoseidonSpecificCols in extensions/native/circuit/src/poseidon2/columns.rs.

New Methods:

• Added a new method write_usize in VmChipTestBuilder in crates/vm/src/arch/testing/mod.rs.

Dependency and Import Changes:

• Removed and reorganized imports related to Poseidon2Opcode and added imports for VerifyBatchOpcode in extensions/native/circuit/src/extension.rs. [1] [2] [3] [4]

[jonathanpwang]

This explanation is probably not going to be that helpful, if we could point to a rust function or a more detailed writeup, that would be better.

[jonathanpwang]

could this be parallelized?

[yi-sun]

I’m not sure where but we are logging “generate Poseidon2 trace” 60K times for an execution of a few microseconds. We should probably remove this.

[jonathanpwang]

Merging to unblock other PRs and because I roughly get the idea.

The following should be done in follow-up:

• Docs (ideally markdown) explaining the constraints in more detail than just the PR description.
  • For example, the top rows are all before the internal rows? The merkle proof compress rows are all adjacent? How exactly are PERM and COMP handled?
• It'd be nice if in the Air code there was more code-level separation (maybe functions) to clarify the big blocks that are only ever enabled for top-level (resp. internal).
• I had a comment about parallelizng trace gen as much as possible, although currently from benchmarks it doesn't seem too much of a concern.

[jonathanpwang]

I’m not sure where but we are logging “generate Poseidon2 trace” 60K times for an execution of a few microseconds. We should probably remove this.

It's from plonky3. fixed in openvm-org/stark-backend#27

[github-actions]

group	app.proof_time_ms	app.cycles	app.cells_used	leaf.proof_time_ms	leaf.cycles	leaf.cells_used
verify_fibair	(-1152 [-33.8%]) 2,258	(-217068 [-30.5%]) 494,614	(-6694890 [-26.2%]) 18,828,444	-	-	-
fibonacci_program	(+48 [+0.8%]) 6,157	1,500,137	51,505,102	(-5759 [-43.0%]) 7,626	(-1078719 [-34.9%]) 2,008,383	(-31460200 [-28.4%]) 79,277,308
regex_program	(-77 [-0.4%]) 18,846	4,190,904	165,028,173	(-12152 [-40.1%]) 18,178	(-2450652 [-41.3%]) 3,483,861	(-69919056 [-28.6%]) 174,238,455
fib_e2e	42,626	12,000,137	410,820,430	54,421	12,345,318	480,146,301
ecrecover_program	(+13 [+0.5%]) 2,622	285,401	15,092,297	(-11084 [-26.9%]) 30,085	(-3988713 [-46.1%]) 4,663,705	(-112692869 [-30.8%]) 253,192,358

Commit: 3c5c4a3

Benchmark Workflow