Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/javascript tracers #6217

Merged
merged 102 commits into from
Dec 1, 2023
Merged

Feature/javascript tracers #6217

merged 102 commits into from
Dec 1, 2023

Conversation

LukaszRozmej
Copy link
Member

@LukaszRozmej LukaszRozmej commented Oct 24, 2023

Changes

  • Adds Javascript tracing to Nethermind

TODO:

  • Integrate ClearScrypt to run JavaScript code
  • Invoke result method
  • Invoke step method
    • Handle step optionality
  • Invoke fault method
  • Invoke enter and exit methods
    • Implement frame parameter
    • Implement result parameter
  • Invoke setup method
    • Make sure the parameter is correctly passed
  • Return results correctly
  • Implement db parameter
  • Implement log parameter
  • Implement stack parameter in optimized way
  • Implement memory parameter
    • Can we do it without copying?
  • implement contract parameter
  • Implement ctx parameter
    • pass blockhash
    • pass error ?
  • Load bigInt
  • Implement toWord function
  • Implement toHex function
  • Implement toAddress function
  • Implement isPrecompiled function
  • Implement slice function
  • Implement toContract function
  • Implement toContract2 function
  • Change arrays to be marshaled to Uint8Array on JS side
  • Potentially - Change how we handle V8 objects - maybe use V8Runtime to not parse all the JS every time from scratch
  • Potentially - optimize the standard trace gathered as it is not needed, maybe change type hierarchies?
  • Properly unit test everything :)
  • Add postStep function to tracer
    • Due to different architecture we are not pre-calculating gasCost and refund before we execute the op-code. To not re-architecture the whole EVM for now we are adding postStep function to tracer where those values are available.
  • Fix fault handling - based on GO code
    • Geth has this weird fault handling:
static OutOfGas -> step with error!

if (dynamicGas)
{
 if memory overflow (dynamic OutOfGas) -> step with error!
 else step
}
else step

do operation
if (error) fault

We currently will not reproduce it, but go with simple:

step
if (error) fault

That means that our step will never have error set to true but we will invoke fault if it errors. Lets try that for now and see how it works, in my opinion it shouldn't produce issues as users have to use defensive programming in step due to weird geth behavior.

Types of changes

What types of changes does your code introduce?

  • Bugfix (a non-breaking change that fixes an issue)
  • New feature (a non-breaking change that adds functionality)
  • Breaking change - potentially breaks existing tracers in plugins
  • Optimization
  • Refactoring
  • Documentation update
  • Build-related changes
  • Other: Description

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

Notes on testing

  • Create a JS tracer that uses all the exposed features.
  • Create a script that will run this tracer on Geth and Nethermind node on last X (might not work past 64-128 blocks due to state pruning) blocks on every transaction. And later will compare the results.

https://github.com/Marchhill/jsTracerTest/tree/main

Documentation

Requires documentation update

  • Yes
  • No

If yes, link the PR to the docs update or the issue with the details labeled docs. Remove if not applicable.

Requires explanation in Release Notes

  • Yes
  • No

If yes, fill in the details here. Remove if not applicable.

Remarks

This PR will possibly replace #6093
Link to API: https://geth.ethereum.org/docs/developers/evm-tracing/custom-tracer

@Marchhill
Copy link
Contributor

Good stuff.

The most important questions:

  • Did we test old-style geth style tracers? (@Marchhill )
  • Can we avoid adding fields to storage-cells?

I've only tested javascript tracers, I will run my script on some old tracers to test them out

Copy link
Contributor

@eliashezron eliashezron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems good to me

@LukaszRozmej LukaszRozmej merged commit 2d76e6d into master Dec 1, 2023
62 checks passed
@LukaszRozmej LukaszRozmej deleted the feature/javascript_tracers branch December 1, 2023 14:10
gr8h pushed a commit to fuseio/nethermind-client that referenced this pull request Dec 12, 2023
* review

* changes

* writing tests

* Import proper packages for all platforms

* Refactoring

* Add built-in tracers

* Hook-up (with incorrect serialization for now!) javascript tracer to JSON RPC

* included more tests

* testing builtings

* Fix loading built-in tracers from files.

Hook-up IWorldTracer.
Fix result json serialization

* merge feature/jstracer

* Writing tests.

* builtIns-implementation

* More conversions

* ready for debug 4byteTracer, Evmdis Tracer, PrestateTrace and Call tracer

* fix test submodule

* fixes

* changes to peek()

* fix test

* further changes

* more changes

* fix getValue in GethJavascriptStyleLog

* More fixes

* fix null

* partial address and stack refactoring

* Improve stack access, fix ctx.to for contracts

* move to TraceMemory

* fix BigInteger by using latest js code

* more improvements

* more changes to make it inline with spec

* fix Ethereum Tests

* fix merge

* Changes and bugfixes

* Make stack, memory and contract not nullable

* Move BigIntegerJS to own class

* Properly hook up Enter & Exit

* Refactors

* simplify conversions

* Move to type arrays

* Move to MemoryTrace for simplicity

* Call fault on actual errors.

Don't call exit for main frame.

Geth logic is weird for faults, we decided to ignore it and always CaptureState on begining with empty error and capture error on end if error occurred.

Geth logic:�StackOverflow -> CaptureState with error!
static OutOfGas -> CaptureState with error!

if (dynamicGas)
{
 if memory overflow -> CaptureState with error!
 else
   actualy capture state!
}
else actually capture state!

do operation
if error Capture Fault

* Fix context and lazy parameters

* fix memory slice

* Make GethLikeTxTrace disposable

* fix NullReferenceException

* Refactor Engine.cs

* Fix db.exist param

* Move to V8Runtime

* Bunch of fixes

* fix slice

* fix BigInteger marshaling

* Add postStep function

* more tests

* fix gasPrice and gas

* Make error in frameresult undefined if needed

* Force serialization of numerals to raw

* clear refund before invoking step

* Populate context more in BlockTracer based on transaction

* fix FrameResult.GasUsed

* fixes

* fix gasCost

* Remove undefined errors

* fix contracts

* Fix wrong values cached in CallFrame & FrameResult

* fix refund as accumulator

* fix tests

* ignore value in STATICCALL

* Allow to access storage slots by hash

* fix contract stack

* exclude "Data" folder

* fix delegate call

* fix clearing state array

* review fixes

* fix

* StorageCell optimizations

* Handle depth for create transactions

* fix DELEGATECALL again

* fix

* Allign errors more with geth

* fix contract address for DELEGATECALL and STATICCALL

* Remove traces from test results

* fix ToContract2

* fix static call

* cosmetic cleanup noticced during review

* Fix revert gas and output

* rename Javascript -> JavaScript

* Change license headers on tracer and link to go-ethereum

* Add comments

* Temporary rename

* Rename `Javascript` to `JavaScript`

* Replace .js files with originals

* Moving BigIntegerJavaScript to seperate file

* Refactor and include .js files in the publishing

---------

Co-authored-by: eliashezron <eliashezron23@gmail.com>
Co-authored-by: MarekM25 <marekm2504@gmail.com>
Co-authored-by: Ruben Buniatyan <rubo@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants