Implementation of EOF in Solidity [DO NOT MERGE] #15294

rodiazet · 2024-07-23T16:52:24Z

EIP-7480 (Only DATALOADN)
EIP-4200 (w/o RJUMPV)
EIP-4750
EIP-6206
EIP-7620
EIP-7069
EIP-5450
evmc update to 12
Change sendMessage interface
Add compileToEOF flag in semantic tests settings
Semantic tests updated

TODO:

There is a couple TODOs added in the commits. They need to be implemented in separated PR.
~~Immutables bug reported by Paradigm needs to be fixed.~~ FIXED
Semantic tests may stop passing because part of them depend on exact contract code which may change with compiler development. Additional support for contract addresses needs to be implemented in soltest.
CI integration may be required.

github-actions · 2024-07-23T16:52:36Z

Thank you for your contribution to the Solidity compiler! A team member will follow up shortly.

If you haven't read our contributing guidelines and our review checklist before, please do it now, this makes the reviewing process and accepting your contribution smoother.

If you have any questions or need our help, feel free to post them in the PR or talk to us directly on the #solidity-dev channel on Matrix.

nikola-matic · 2024-07-30T10:17:21Z

libevmasm/Assembly.cpp

-		m_items.back().setLocation(m_currentSourceLocation);
-	m_items.back().m_modifierDepth = m_currentModifierDepth;
-	return m_items.back();
+	auto& currentItems = m_codeSections.at(m_currentCodeSection).items;


m_codeSection.at(...) can throw - either a util::contains check should go here, or even better, an assertion if you're sure that the current code section is present.

Right. I'm adding an assertion as m_currentCodeSection is changed on in void beginFunction(uint16_t _functionID) and similar assertion is there too.

nikola-matic · 2024-07-30T10:22:05Z

libevmasm/Assembly.cpp

 	for (unsigned tagSize = subTagSize; true; ++tagSize)
 	{
 		size_t ret = 1;
-		for (auto const& i: m_data)


Is m_data no longer taken into account?

That part of the calculation moved to ::assemble - it's part of bytesRequiredForDataUpperBound there (both for EOF and legacy)...

But we should probably rethink this as well at the latest as part of #15294 (comment) - For example: since in EOF there won't be any PushTags, the weird outer loop here becomes superfluous as well, since code size no longer depends on the tag size (the "tag size" in general becomes irrelevant).

For now, maybe a comment in the header is enough, clarifying that even for legacy this now only counts "pure" code size without data?

libevmasm/Assembly.cpp

nikola-matic · 2024-07-30T16:34:06Z

libevmasm/Assembly.cpp

+
+		std::vector<size_t> successors;
+
+		if (


I assume that ConditionalRelativeJump is not here because both 'branches' need to have their stack heights calculated? I.e. both the next assembly item as well as the tagged one that we're (possibly) jumping to?

Yes. For ConditionalRelativeJump we need to calculated both branches and one of them starts with the next instruction but second is the jump target. It's being added to successors by next if

nikola-matic · 2024-07-30T16:38:57Z

libevmasm/Assembly.cpp

+{
+	uint16_t maxStackHeight = _args;
+	std::stack<size_t> worklist;
+	std::vector<int32_t> stack_heights(_items.size(), -1);


Suggested change

std::vector<int32_t> stack_heights(_items.size(), -1);

std::vector<size_t> stack_heights(_items.size(), std::numeric_limits<size_t>::max);

stackHeight should also be unsigned - I would assume our stack height is never going to be negative?

libevmasm/Assembly.cpp

libevmasm/BlockDeduplicator.cpp

ekpyron · 2024-07-31T21:06:33Z

libevmasm/Assembly.cpp

 	Functionalizer f(_out, _prefix, _sourceCodes, *this);

-	for (auto const& i: m_items)
+	// TODO: support EOF


In cases like this (and e.g. similarly for Assembly::assemblyJSON and possibly a few of other similar TODOs like this), it'd be better to add an explicit

solUnimplementedAssert(!m_eofVersion.has_value(), "Assembly output for EOF is not yet implemented.");

Better to error out and then implement it properly later in a subsequent PR than to give a partial output of only the first code section with the rest missing.

Done. Additionally I switched off the assembly optimization for EOF as event first step requires verification.

…typo in input parsing Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

…EIP-3540) Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

Add `DATALOAD` instruction

Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

Preparation to implement new contract creation Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

…endMessage Co-authored-by: Rodrigo Q. Saramago <rodrigoqsaramago@gmail.com>

libsolidity/experimental/codegen/IRGenerator.cpp

ekpyron · 2024-08-22T15:05:45Z

libyul/backends/evm/ControlFlowGraphBuilder.cpp

 			return !reachabilityCheck.visited.count(entry);
 		});
+
+	// Remove functions which are never referenced.


This change in cleanUnreachable could be split out into a separate PR and merged directly into develop up front, right?
EDIT: we can revisit once we get to the commits containing it, whether splitting really makes things easier, fine for now.

ekpyron · 2024-08-22T15:21:28Z

libsolidity/interface/StandardCompiler.cpp

 		if (!settings["eofVersion"].is_number_unsigned())
 			return formatFatalError(Error::Type::JSONError, "eofVersion must be an unsigned integer.");
-		auto eofVersion = settings["evmVersion"].get<uint8_t>();
+		auto eofVersion = settings["eofVersion"].get<uint8_t>();


This and the related evm-version check change can and should separately go into develop :-).
EDIT: as per chat, let's just merge the first three commits for now :-).

ekpyron · 2024-08-22T16:08:43Z

libsolidity/interface/CompilerStack.cpp

 	Contract& compiledContract = m_contracts.at(_contract.fullyQualifiedName());

-	std::shared_ptr<Compiler> compiler = std::make_shared<Compiler>(m_evmVersion, m_revertStrings, m_optimiserSettings);
+	std::shared_ptr<Compiler> compiler = std::make_shared<Compiler>(m_evmVersion, m_eofVersion, m_revertStrings, m_optimiserSettings);


Instead of passing m_eofVersion around here, we can just assert

solUnimplementedAssert(!m_eofVersion, "EOF is not supported for legacy code generation");

(this is only-legacy, see the solAssert(!m_viaIR, ""); below)
And thereby leave libsolidity/codegen/Compiler.h and libsolidity/codegen/CompilerContext.h largely untouched (resp. where they need an eof version just use std::nullopt directly).

Addressed in https://github.com/ethereum/solidity/pull/15356/files#diff-0f19df92d55a7d7fe38fdf4be1e14b364671c4934853d13a33b31aedf3f917bdR1424

cairoeth · 2024-08-23T05:13:48Z

libevmasm/AssemblyItem.cpp

+		text = "eofcreate(" +  std::to_string(static_cast<size_t>(data())) + ")";
+		break;
+	case ReturnContract:
+		text = "returcontract(" +  std::to_string(static_cast<size_t>(data())) + ")";


typo in assembly text of RETURNCONTRACT

Suggested change

text = "returcontract(" + std::to_string(static_cast<size_t>(data())) + ")";

text = "returncontract(" + std::to_string(static_cast<size_t>(data())) + ")";

rodiazet · 2024-11-27T15:17:18Z

This PR is splitted into many separated PRs which are in the review. This PRs is supposed to be closed without merging after those PRs are merged.

github-actions bot added the external contribution ⭐ label Jul 23, 2024

rodiazet force-pushed the eof-cleaned branch 10 times, most recently from c49b88b to 88cf606 Compare July 29, 2024 16:56

nikola-matic reviewed Jul 31, 2024

View reviewed changes

ekpyron reviewed Jul 31, 2024

View reviewed changes

ekpyron mentioned this pull request Jul 31, 2024

Refine and Stabilize EOF Support #15310

Open

71 tasks

rodiazet force-pushed the eof-cleaned branch 2 times, most recently from 5ab6923 to 64e4050 Compare August 8, 2024 07:57

rodiazet and others added 13 commits August 8, 2024 15:14

prep: Extend StandardCompiler settings with eofVersion flag. Fix …

4ae6d1c

…typo in input parsing Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

prep: Pass _eofVersion to libevmasm/Assembly class

edc3b40

Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

prep: Change toBigEndian interface to support rvalue ref.

07528fc

Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

eof: Introduce EOF container format support in Assembly::assemble (…

c0a839e

…EIP-3540) Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

eof: Support DATALOADN (EIP-7480)

84ab885

Add `DATALOAD` instruction

eof: Support RJUMP, RJUMPI (EIP-4200)

70b9e8b

Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

eof: Support CALLF, JUMPF, RETF. (EIP-4750, EIP-6206)

64cbc1d

Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

eof: Pass flag to copyConstructorArgumentsToMemoryFunction (EIP-7620)

412edcd

Preparation to implement new contract creation Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

eof: Implement EOF Contract Creation (EIP-7620)

76a25cc

eof: Implement revamped CALL instructions (EIP-7069)

a419908

eof: Implement max stack heigh calculation (EIP-5450)

79b4d65

Co-authored-by: Daniel Kirchner <daniel@ekpyron.org>

prep: Update to emvc 12

bdc126f

tests: Add new (EOF) message kind to EVMHost. Separate arguments in s…

00b0b14

…endMessage Co-authored-by: Rodrigo Q. Saramago <rodrigoqsaramago@gmail.com>

rodiazet added 2 commits August 8, 2024 15:40

tests: Add compileToEOF flag to semantic tests settings

a902291

tests: Update semantic tests for EOF

bbfb85d

rodiazet force-pushed the eof-cleaned branch from 64e4050 to bbfb85d Compare August 8, 2024 13:46

ekpyron reviewed Aug 22, 2024

View reviewed changes

libsolidity/experimental/codegen/IRGenerator.cpp Show resolved Hide resolved

ekpyron reviewed Aug 22, 2024

View reviewed changes

cairoeth reviewed Aug 23, 2024

View reviewed changes

cameel added the EOF label Aug 29, 2024

rodiazet changed the title ~~Implementation of EOF in Solidity~~ Implementation of EOF in Solidity [DO NOT MERGE] Nov 27, 2024

rodiazet closed this Dec 11, 2024

	std::vector<int32_t> stack_heights(_items.size(), -1);
	std::vector<size_t> stack_heights(_items.size(), std::numeric_limits<size_t>::max);

	text = "returcontract(" + std::to_string(static_cast<size_t>(data())) + ")";
	text = "returncontract(" + std::to_string(static_cast<size_t>(data())) + ")";

Implementation of EOF in Solidity [DO NOT MERGE] #15294

Implementation of EOF in Solidity [DO NOT MERGE] #15294

Uh oh!

Conversation

rodiazet commented Jul 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 23, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rodiazet Aug 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ekpyron Aug 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ekpyron Aug 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ekpyron Aug 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rodiazet commented Nov 27, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

rodiazet commented Jul 23, 2024 •

edited

Loading

rodiazet Aug 7, 2024 •

edited

Loading

ekpyron Aug 22, 2024 •

edited

Loading

ekpyron Aug 22, 2024 •

edited

Loading

ekpyron Aug 22, 2024 •

edited

Loading