core/evm: avoid copying memory for input in calls #20177

holiman · 2019-10-17T08:12:52Z

This PR avoids copying memory during calls, and instead passes memory slices by reference. The input in execution should not ever be modified, but I'm not 100% sure that this is always respected (particularly I haven't looked in depth into whether alll of our precompiles respects this -- EDIT: ecRecover did not, fixed).

If this PR works out, it might be worth it to add a new type type ReadonlyMem []byte, and pass that around instead of a raw slice, to make it more obvious that this type should not be modified.

This PR is (so far) mainly for benchmarking, and validating that we can indeed sync through the history without any hiccups.

holiman · 2019-10-17T08:30:56Z

Running on benchmarkers now: https://geth-bench.ethdevops.io/d/Jpk-Be5Wk/dual-geth?orgId=1&refresh=30s&var-exp=mon08&var-master=mon09&var-percentile=50&from=now-1h&to=now

holiman · 2019-10-17T09:43:01Z

Travis found a bug, where ecRecover would actually modify the input buffer. Fixed now, tests added, restarted the benchmarker.

core/vm/memory.go

holiman · 2019-10-17T11:35:03Z

Lifting to top level:

I'm not really a fan of this. []byte doesn't have any methods itself, so aliasing won't hide anything. Calling code will still have access to all the slice functions and indexing and modification. I think this name just gives a false sense of security or confusion.

So, this PR introduces the invariant that we don't modify input on call derivates. This is really a no-brainer, because it's already set in the protocol: hence why call derivates do not charge per byte of calldata, since it's assumed that no new allocations are required.

However, since we do need to ensure that there's no code path where we ever violate that, like ecRecover did, and it would be neat if we could somehow enforce it. However, I agree that my weak attempt with aliasing might be totally off the mark.

karalabe · 2019-10-17T11:39:17Z

Perhaps we could add a fuzzer that would pick all the opcodes and try to run them on random input. If we hook that into a contiguous fuzzer (like fuzzit), it might be a good way to catch accidents.

karalabe · 2019-10-17T11:43:23Z

core/vm/memory.go

 // GetPtr returns the offset + size
-func (m *Memory) GetPtr(offset, size int64) []byte {
+func (m *Memory) GetPtr(offset, size int64) ReadOnlyMem {


Do we still use the Get version? If not, perhaps lets get rid of it to avoid reintroducing the copies later?

Yes,

Log operations,

Create2 and Create,

Mload

.... that being said.. it's useless to have Mload use Get as opposed to GetPtr, since all it does is send it into SetBytes on a big.Int, which copies it internally...

…crecover not modify input buffer

holiman · 2019-10-23T13:45:58Z

The benchmark run has now completed 8.2M blocks of full-sync, so looking good

holiman · 2019-10-24T15:49:15Z

Full sync to 8.8M completed (without the last commit, which was added later).

fjl

I think this change is really good, but the memory API could be improved to highlight the difference between Get and GetPtr more. We should rename Get -> GetCopy and GetPtr -> Get.

holiman · 2019-10-30T11:43:28Z

I think this change is really good, but the memory API could be improved to highlight the difference between Get and GetPtr more. We should rename Get -> GetCopy and GetPtr -> Get.

I can agree about Get->GetCopy, but I think we should leave GetPtr as is, since it's more indicative of the fact that you're getting a pointer (well, an existing slice)

holiman · 2019-10-30T11:46:37Z

Done, ptal

karalabe · 2019-10-30T12:04:51Z

core/vm/instructions.go

@@ -725,7 +723,7 @@ func opCreate2(pc *uint64, interpreter *EVMInterpreter, contract *Contract, memo
 		endowment    = stack.pop()
 		offset, size = stack.pop(), stack.pop()
 		salt         = stack.pop()
-		input        = memory.Get(offset.Int64(), size.Int64())
+		input        = memory.GetCopy(offset.Int64(), size.Int64())


Why do we need copies in create? I know I asked this already, sorry, don't remember :P

I'm not 100% sure we need it, but Create/Create2 holds on to that code, and stores it in a struct for later processing. I didn't consider it safe to change that now

karalabe

Small question, otherwise LGTM

* core/evm, contracts: avoid copying memory for input in calls + make ecrecover not modify input buffer * core/vm: optimize mstore a bit * core/vm: change Get -> GetCopy in vm memory access

@jpeletier

* core: log chain reorg/split metrics (ethereum#18950) * core: log chain reorg/split metrics * core: report 1-block reorgs on the metrics too * common: unify hex prefix check code (ethereum#19937) * ethdb/dbtest: addd test suite for ethdb backends (ethereum#19960) - Move the existing tests from memorydb into a generalized testsuite that can be run by any ethdb backend implementation. - Add several more test cases to clarify some non-obvious nuances when implementing a custom ethdb backend, such as the behaviour of NewIteratorWithPrefix vs NewIteratorWithStart. - Add leveldb to the testsuite using in-memory storage for fast execution. * core/vm: enable istanbul EIPs in the jump table * build: gomobile automaticall adds the ios tag, don't duplicate * cmd/utils: customize cli.HelpPrinter to fix alignment (ethereum#19956) This copies cli.printHelp but changes minwidth to 38. Custom flag code is improved to print the default value using cli.FlagStringer like all built-in flags do. * p2p/enode: allow DNS names in enode URLs (ethereum#18524) * crypto: add SignatureLength constant and use it everywhere (ethereum#19996) Original change by @jpeletier * rpc: enable compression on HTTP transport (ethereum#19997) This change adds support for gzip encoding on HTTP responses. Gzip encoding is used when the client sets the 'accept-encoding: gzip' header. Original change by @brianosaurus, with fixes from @SjonHortensius. * retesteth: implement istanbul support * tests: implement Istanbul support * acmd, core, eth, les: support --override.istanbul * README: change chainID to <arbitrary positive integer> (ethereum#20002) * Change chainId in genesis block to 8888 from 0 Change chainId in genesis block to 8888 from 0 per Moriteru in https://ethereum.stackexchange.com/a/28082/40230. * Replace 8888 with “<arbitrary positive integer>” Per PR review, replace Moriteru’s arbitrary positive integer 8888 with “<arbitrary positive integer>” in chainId field. * les: wait for all task goroutines before dropping the peer (ethereum#20010) * les: wait all task routines before drop the peer * les: address comments * les: fix issue * les: fix panic (ethereum#20013) * eth: disallow overwrite files via admin.exportChain * retesteth: enable maxResults in AccountRange (ethereum#20020) * rpc: raise limit in TestClientNotificationStorm (ethereum#19999) * cmd/utils: reduce light.maxpeers default for clients to 1/10th (ethereum#19933) Currently light.maxpeers is 100 - after this change it's 10 for non-servers. Fixes ethereum#19820 * params: release Geth v1.9.3 stable * params: begin Geth v1.9.4 release cycle * Dockerfile: expose GraphQL ports * README: accounts in alloc field should exist (ethereum#20005) * Accounts in alloc field must already exist Note that accounts in alloc field must already exist, as pointed out by Simeon Vanov in https://gettoshare.com/2017/10/30/how-to-use-genesis-json-alloc-property/ * Change wording per PR review comment * README: minor fixups * common, graphql: fix hash/address decoding + UI content type * core, metrics, p2p: switch some invalid counters to gauges * core/state: optimize some internals during encoding * eth: remove unused field (ethereum#20049) * tests: expose internal RunNoVerify method (ethereum#20051) * all: make unit tests work with Go 1.13 (ethereum#20053) Most of these changes are related to the Go 1.13 changes to test binary flag handling. * cmd/geth: make attach tests more reliable This makes the test wait for the endpoint to come up by polling it instead of waiting for two seconds. * tests: fix test binary flags for Go 1.13 Calling flag.Parse during package initialization is prohibited as of Go 1.13 and causes test failures. Call it in TestMain instead. * crypto/ecies: remove useless -dump flag in tests * p2p/simulations: fix test binary flags for Go 1.13 Calling flag.Parse during package initialization is prohibited as of Go 1.13 and causes test failures. Call it in TestMain instead. * build: remove workaround for ./... vendor matching This workaround was necessary for Go 1.8. The Go 1.9 release changed the expansion rules to exclude vendored packages. * Makefile: use relative path for GOBIN This makes the "Run ./build/bin/..." line look nicer. * les: fix test binary flags for Go 1.13 Calling flag.Parse during package initialization is prohibited as of Go 1.13 and causes test failures. Call it in TestMain instead. * travis, Dockerfile, appveyor: bump to Go 1.13 * build: switch PPA from Gophers dep to manual download * Revert "build: switch PPA from Gophers dep to manual download" (ethereum#20061) * core: remove unused gas return in ApplyTransaction (ethereum#20065) * rlp: improve nil pointer handling (ethereum#20064) * rlp: improve nil pointer handling In both encoder and decoder, the rules for encoding nil pointers were a bit hard to understand, and didn't leave much choice. Since RLP allows two empty values (empty list, empty string), any protocol built on RLP must choose either of these values to represent the null value in a certain context. This change adds choice in the form of two new struct tags, "nilString" and "nilList". These can be used to specify how a nil pointer value is encoded. The "nil" tag still exists, but its implementation is now explicit and defines exactly how nil pointers are handled in a single place. Another important change in this commit is how nil pointers and the Encoder interface interact. The EncodeRLP method was previously called even on nil values, which was supposed to give users a choice of how their value would be handled when nil. It turns out this is a stupid idea. If you create a network protocol containing an object defined in another package, it's better to be able to say that the object should be a list or string when nil in the definition of the protocol message rather than defining the encoding of nil on the object itself. As of this commit, the encoding rules for pointers now take precedence over the Encoder interface rule. I think the "nil" tag will work fine for most cases. For special kinds of objects which are a struct in Go but strings in RLP, code using the object can specify the desired encoding of nil using the "nilString" and "nilList" tags. * rlp: propagate struct field type errors If a struct contained fields of undecodable type, the encoder and decoder would panic instead of returning an error. Fix this by propagating type errors in makeStruct{Writer,Decoder} and add a test. * cmd/evm: make evm default to all ethash protocol changes * core/state: accumulate writes and only update tries when must * core: add blockchain test too for revert cornercase * common/mclock: clean up AfterFunc support (ethereum#20054) This change adds tests for the virtual clock and aligns the interface with the time package by renaming Cancel to Stop. It also removes the binary search from Stop because it complicates the code unnecessarily. * core: smaller txpool status locking (ethereum#20080) * txpool: smaller lock portion * core/tx_pool: fix data race * core: dedup known transactions without global lock, track metrics * les: multiple server bugfixes (ethereum#20079) * les: detailed relative cost metrics * les: filter txpool relative request statistic * les: initialize price factors * les: increased connected bias to lower churn rate * les: fixed clientPool.setLimits * core: do not use mutex in GetAncestor * les: bump factor db version again * les: add metrics * les, light: minor fixes * core: fix tx dedup return error count * params: activate Istanbul on Ropsten and Görli * params: bump CHTs for the 1.9.4 release * core/forkid, params: fix tests, enable Istanbul on Rinkeby + testers * vendor: pull in USB Windows fixes * params: release Geth v1.9.4 stable * params: start v1.9.5 release cycle * params: remove legacy bootnodes * core/state: fix state object deep copy (ethereum#20100) deepCopy didn't copy pending storage updates, leading to the creation of blocks with invalid state root. * params: release Geth v1.9.5 stable * params: start v1.9.6 release cycle * core/state: fix copy-commit-copy (ethereum#20113) * core/state: revert noop finalise, fix copy-commit-copy * core/state: reintroduce net sstore tracking, extend tests for it * dashboard: log host+port * les: fix checkpoint sync (ethereum#20120) * p2p/dnsdisc: add implementation of EIP-1459 (ethereum#20094) This adds an implementation of node discovery via DNS TXT records to the go-ethereum library. The implementation doesn't match EIP-1459 exactly, the main difference being that this implementation uses separate merkle trees for tree links and ENRs. The EIP will be updated to match p2p/dnsdisc. To maintain DNS trees, cmd/devp2p provides a frontend for the p2p/dnsdisc library. The new 'dns' subcommands can be used to create, sign and deploy DNS discovery trees. * tests/solidity: add contract to test every opcode (ethereum#19283) Fixes ethereum#18210 * internal/ethapi: support block number or hash on state-related methods (ethereum#19491) This change adds support for EIP-1898. * core/blockchain: remove block from futureBlocks on error (ethereum#19763) * core/state: fix database leak and copy tests (ethereum#19306) * core: initialize current block/fastblock atomics to nil, fix ethereum#19286 (ethereum#19352) * ethdb/leveldb: disable seek compaction (ethereum#20130) * vendor: update leveldb * ethdb/leveldb: disable seek compaction and add metrics * vendor: udpate latest levledb * ethdb/leveldb: fix typo * p2p: measure subprotocol bandwidth usage * github: Added capital P (ethereum#20139) * tests: update test suite for istanbul (ethereum#20082) * update tests for istanbul * tests: updated blockchaintests, see ethereum/tests#637 * tests: update again, hopefully fixed this time * tests: skip time consuming, run legacy tests * tests: update again * build: disable long-running tests on travis * tests: fix formatting nits * tests: I hate github's editor * cmd/bootnode: fix exit behavior with -genkey (ethereum#20110) * les: add empty "les" ENR entry for servers (ethereum#20145) * params: release Geth v1.9.6 stable * params: begin v1.9.7 release cycle * cmd/utils: fix command line flag resolve (ethereum#20167) In Geth, we have two sources for configuration: (1) Config file (2) Command line flag Basically geth will first resolve config file and then overwrite configs with command line flags. This issue is: geth should only overwrite configs if flags are truly set. So before we apply any flag to configs, `GlobalIsSet` check is necessary. * params: check fork ordering when initializing new genesis, fixes ethereum#20136 (ethereum#20169) prevent users from misconfiguring their nodes so that fork ordering is not preserved. * p2p/dnsdisc: update to latest EIP-1459 spec (ethereum#20168) This updates the DNS TXT record format to the latest changes in ethereum/EIPs#2313. * p2p/simulations: add node properties support and utility functions (ethereum#20060) * README: use new miner threads flag instead of legacy minerthreads flag (ethereum#20165) * Changed http:// to https:// on links in log/README.md (ethereum#20178) docs: change http to https on links in log/README.md * dashboard: change links in README to https (ethereum#20181) Changed http:// to https:// on links in dashboard/README.md * metrics: change links in README.md to https (ethereum#20182) * miner: add generate and import unit test (ethereum#20111) This PR adds a new unit test in miner package which will create some blocks from miner and then import into another chain. In this way, we can ensure all blocks generated by Geth miner obey consensus rules. * accounts/abi/bind: take into account gas price during gas estimation (ethereum#20189) The gas price was not passed to the `EstimateGas` function. As a result, conditional execution paths depending on `tx.gasprice` could be not correctly processed and we could get invalid gas estimates for contract function calls. * clef: resolve windows pipes, fixes ethereum#20121 (ethereum#20166) * consensus: fix possessives in comments. (ethereum#20209) * cmd/evm: remove surrounding whitespace in hex input code (ethereum#20211) This way, the output of `evm compile` can be used directly in `evm --codefile code.txt run`, without stripping the trailing newline first. * trie: remove node ordering slice in sync batch (ethereum#19929) When we flush a batch of trie nodes into database during the state sync, we should guarantee that all children should be flushed before parent. Actually the trie nodes commit order is strict by: children -> parent. But when we flush all ready nodes into db, we don't need the order anymore since (1) they are all ready nodes (no more dependency) (2) underlying database provides write atomicity * core/asm: assembly parser label fixes (ethereum#20210) * core/asm: Fix encoding of pushed labels EVM uses big-endian byte-order, so to pad a label value to 4 bytes, zeros must be added to the front, not the end. * core/asm: Fix PC calculations when a label is pushed Incrementing PC by 5 is only correct if the label appears after a jump, in which case there is an implicit push. When it appears after an explicit push, PC should only be incremented by 4. * core/asm: Allow JUMP with no argument This way, a label can be pushed explicitly, or loaded from memory to implement a jump table. * eth/downloader: fix data race in downloader * cmd/devp2p, p2p: dial using node iterator, discovery crawler (ethereum#20132) * p2p/enode: add Iterator and associated utilities * p2p/discover: add RandomNodes iterator * p2p: dial using iterator * cmd/devp2p: add discv4 crawler * cmd/devp2p: WIP nodeset filter * cmd/devp2p: fixup lesFilter * core/forkid: add NewStaticFilter * cmd/devp2p: make -eth-network filter actually work * cmd/devp2p: improve crawl timestamp handling * cmd/devp2p: fix typo * p2p/enode: fix comment typos * p2p/discover: fix comment typos * p2p/discover: rename lookup.next to 'advance' * p2p: lower discovery mixer timeout * p2p/enode: implement dynamic FairMix timeouts * cmd/devp2p: add ropsten support in -eth-network filter * cmd/devp2p: tweak crawler log message * eth: eth/64 - extend handshake packet with fork id * core/forkid: add two clauses for more precise validation (ethereum#20220) * miner: increase import time allowance (ethereum#20217) Fix the block import unit test which can time out sometimes. * cmd/devp2p, core/forkid: make forkid.Filter API uniform * params, core/forkid: configure mainnet istanbul block 9069K (ethereum#20222) * params: configure mainnet istanbul block 9069K * core/forkid: add some more test items for mainnet istanbul * accounts/abi: add internalType information and fix issues (ethereum#20179) * accounts/abi: fix various issues The fixed issues include: (1) If there is no return in a call function, unpack should return nil error (2) For some functions which have struct array as parameter, it will also be detected and generate the struct definition (3) For event, if it has non-indexed parameter, the parameter name will also be assigned if empty. Also the internal struct will be detected and generate struct defition if not exist. (4) Fix annotation generation in event function * accounts/abi: add new abi field internalType * accounts: address comments and add tests * accounts/abi: replace strings.ReplaceAll with strings.Replace * appveyor: bump to Go 1.13.4 * les: rework clientpool (ethereum#20077) * les: rework clientpool * cmd/puppeth: integrate istanbul into puppeth (ethereum#19926) * cmd/puppeth: integrate istanbul into puppeth * cmd/puppeth: address comment * cmd/puppeth: use hexutil.Big for fork indicator * cmd/puppeth: finalize istanbul fork * cmd/puppeth: fix 2200 for parity, rename is to eip1283ReenableTransition * cmd/puppeth: fix eip1108 * cmd/puppeth: add blake2f for parity * cmd/puppeth: add aleth istanbul precompiled * cmd/puppeth: use hexutil.Big * cmd/puppeth: fix unit tests * cmd/puppeth: update testdata * core/evm: avoid copying memory for input in calls (ethereum#20177) * core/evm, contracts: avoid copying memory for input in calls + make ecrecover not modify input buffer * core/vm: optimize mstore a bit * core/vm: change Get -> GetCopy in vm memory access * travis, build, internal: use own Go bundle for PPA builds (ethereum#20240) * build: bump PPAs to Go 1.13 (via longsleep), keep Trusty on 1.11 * travis, build, vendor: use own Go bundle for PPA builds * travis, build, internal, vendor: smarter Go bundler, own untar * build: updated ci-notes with new Go bundling, only make, don't test * travis: bump linter to Go 1.13.x * params: hard-code new CHTs for the 1.9.7 release * les: fix and slim the unit tests of les (ethereum#20247) * les: loose restriction of unit tests * les: update unit tests * les, light: slim the unit tests * params: release Geth v1.9.7 * params: begin v1.9.8 release cycle * p2p/enode: mock DNS resolver in URL parsing test (ethereum#20252) * travis: enable test suite on ARM64 (ethereum#20219) * travis: Enable ARM support * Include fixes from 20039 * Add a trace to debug the invalid lookup issue * Try increasing the timeout to see if the arm test passes * Investigate the resolver issue * Increase arm64 timeout for clique test * increase timeout in tests for arm64 * Only test the failing tests * Review feedback: don't export epsilon * Remove investigation tricks+include fjl's feeback * Revert the retry ahead of using the mock resolver * Fix rebase errors * p2p: fix bug in TestPeerDisconnect (ethereum#20277) * dashboard: send current block to the dashboard client (ethereum#19762) This adds all dashboard changes from the last couple months. We're about to remove the dashboard, but decided that we should get all the recent work in first in case anyone wants to pick up this project later on. * cmd, dashboard, eth, p2p: send peer info to the dashboard * dashboard: update npm packages, improve UI, rebase * dashboard, p2p: remove println, change doc * cmd, dashboard, eth, p2p: cleanup after review * dashboard: send current block to the dashboard client * miner: increase worker test timeout (ethereum#20268) TestEmptyWork* occasionally fails due to timeout. Increase the timeout. * les: implement server priority API (ethereum#20070) This PR implements the LES server RPC API. Methods for server capacity, client balance and client priority management are provided. * accounts/abi/bind, cmd/abigen: implement alias for abigen (ethereum#20244) * accounts/abi/bind, cmd/abigen: implement alias for abigen * accounts/abi/bind: minor fixes * accounts/abi/bind: address comments * cmd/abigen: address comments * accounts/abi/bind: print error log when identifier collision * accounts/abi/bind: address comments * accounts/abi/bind: address comment * rpc: fix typo example code (ethereum#20284) * cmd/faucet: use github.com/gorilla/websocket (ethereum#20283) golang.org/x/net/websocket is unmaintained, and we have already switched to using github.com/gorilla/websocket for package rpc. * dashboard: remove the dashboard (ethereum#20279) This removes the dashboard project. The dashboard was an experimental browser UI for geth which displayed metrics and chain information in real time. We are removing it because it has marginal utility and nobody on the team can maintain it. Removing the dashboard removes a lot of dependency code and shaves 6 MB off the geth binary size. * whisper/whisperv6: fix staticcheck issues (ethereum#20288) * build: gather info to investigate why builds fail on ARM (ethereum#20281) * params: finish sentence in comment (ethereum#20291) * core/vm: fix tracer interface parameter name (ethereum#20294) * internal/ethapi: don't query wallets at every execution of gas estimation * cmd/evm: Allow loading input from file (ethereum#20273) Make it possible to load input from a file. Simlar to `--code` / `--codefile`, have `--input`/`--inputfile`. * rpc, p2p/simulations: use github.com/gorilla/websocket (ethereum#20289) * rpc: improve codec abstraction rpc.ServerCodec is an opaque interface. There was only one way to get a codec using existing APIs: rpc.NewJSONCodec. This change exports newCodec (as NewFuncCodec) and NewJSONCodec (as NewCodec). It also makes all codec methods non-public to avoid showing internals in godoc. While here, remove codec options in tests because they are not supported anymore. * p2p/simulations: use github.com/gorilla/websocket This package was the last remaining user of golang.org/x/net/websocket. Migrating to the new library wasn't straightforward because it is no longer possible to treat WebSocket connections as a net.Conn. * vendor: delete golang.org/x/net/websocket * rpc: fix godoc comments and run gofmt * build: use golangci-lint (ethereum#20295) * build: use golangci-lint This changes build/ci.go to download and run golangci-lint instead of gometalinter. * core/state: fix unnecessary conversion * p2p/simulations: fix lock copying (found by go vet) * signer/core: fix unnecessary conversions * crypto/ecies: remove unused function cmpPublic * core/rawdb: remove unused function print * core/state: remove unused function xTestFuzzCutter * core/vm: disable TestWriteExpectedValues in a different way * core/forkid: remove unused function checksum * les: remove unused type proofsData * cmd/utils: remove unused functions prefixedNames, prefixFor * crypto/bn256: run goimports * p2p/nat: fix goimports lint issue * cmd/clef: avoid using unkeyed struct fields * les: cancel context in testRequest * rlp: delete unreachable code * core: gofmt * internal/build: simplify DownloadFile for Go 1.11 compatibility * build: remove go test --short flag * .travis.yml: disable build cache * whisper/whisperv6: fix ineffectual assignment in TestWhisperIdentityManagement * .golangci.yml: enable goconst and ineffassign linters * build: print message when there are no lint issues * internal/build: refactor download a bit * travis: remove traces and use travis_wait in ARM build (ethereum#20296) * travis: remove debug traces * travis: Add travis_wait to the test run * travis: increase travis_wait time * core: s/isEIP155/isHomestead/g (fix IntrinsicGas signature var name) (ethereum#20300) * core: s/isEIP155/isEIP2/ (fix) This signature variable name reflects a spec'd change in gas cost for creating contracts as documented in EIP2 (Homestead HF). https://github.com/ethereum/EIPs/blob/master/EIPS/eip-2.md#specification * core: s/isEIP2/sIsHomestead/g Use isHomestead since Homestead is what the caller and rest of the code uses. * les: rename UpdateBalance to AddBalance and simplify return format (ethereum#20304) * travis: use travis_wait for both install and build (ethereum#20309) * build: add test cmd flag -v for verbose logs (ethereum#20298) Adds flags akin to -coverage flag enabling the test runner to use go test's -v flag, signaling verbose test log output. * cmd/clef: fix staticcheck warnings (ethereum#20314) * consensus/clique: add clique_status API method (ethereum#20103) This PR introduces clique_status which gives info about the health of the clique network. It's currently a bit PITA to find out how a clique network is performing, and it can easily happen that sealers drop off -- and everything is 'fine' until one more signer drops off, and the network suddenly halts. The new method provides the following stats: - Which signers are currently active, and have signed blocks in the last N (set to 64) blocks? - How many blocks has each signer signed? - What is the difficulty in the last N blocks, compared to the theoretical maximum? * consensus/clique: fix struct tags for status API (ethereum#20316) Also unexport the status struct. * cmd/ethkey: fix file permissions in changepassword command (ethereum#20313) Found by staticcheck. * p2p/netutil: fix staticcheck warning (ethereum#20315) * core/rawdb: check hash before return data from ancient db (ethereum#20195) * core/rawdb: check hash before return data from ancient db * core/rawdb: fix lint * core/rawdb: calculate the hash in the fly * travis: deactivate arm build during push (ethereum#20321) * ethclient: remove use of common.ToHex (ethereum#20326) * cmd/wnode: remove uses of common.ToHex (ethereum#20327) * event: remove unused field 'closed' (ethereum#20324) * .github: remove 'nonsense' from CODEOWNERS (ethereum#20329) * whisper/whisperv6: fix staticcheck warnings (ethereum#20328) * p2p: remove unused code (ethereum#20325) * p2p/simulations: fix staticcheck warnings (ethereum#20322) * internal/web3ext, les: update clique JS and make it work with the light client (ethereum#20318) Also fix the input formatter on clique_getSnapshot and clique_getSigners so that integers as well as hex number strings are accepted. * rpc: remove 'exported or builtin' restriction for parameters (ethereum#20332) * rpc: remove 'exported or builtin' restriction for parameters There is no technial reason for this restriction because package reflect can create values of any type. Requiring parameters and return values to be exported causes a lot of noise in package exports. * rpc: fix staticcheck warnings * core: fix staticcheck warnings (ethereum#20323) * miner: fix data race in tests (ethereum#20310) * miner: fix data race in tests miner: fix linter * miner: address comment * cmd/puppeth: update chain spec of parity (ethereum#20241) * go.mod, vendor: switch to Go modules * travis: explicitly enable go modules in Go 1.11 and 1.12 * accounts/abi/bind: switch binding test to go modules * travis, build: aggregate and upload go mod dependencies for PPA * go.mod: tidy up the modules to avoid xgo writes to go.sum * build, internal/build: drop own file/folder copier * travis: fake build ppa only for go module dependencies * mobile: fix CopyFile switch to package cp * tests: refactor TestState to dedupe walk callback Minor refactoring. * build, travis: use ephemeral debsrc GOPATH to get mod deps * tests: enable TransactionTests Istanbul case (ethereum#20337) * cmd/puppeth: x-spec nonce data type, use types.BlockNonce Refactors to use existing BlockNonce type instead of hand-rolled bytes logic. * cmd/puppeth: make ssh prompt more user-friendly * common/hexutil: improve GraphQL error messages (ethereum#20353) * build: pull in ci.go dependencies for the PPA builder * common: improve GraphQL error messages (ethereum#20354) * core/types: remove BlockBy sorting code (ethereum#20355) * build: skip go clean on PPA, messes with the module trick * accounts/abi/bind/backends: remove unused assignment (ethereum#20359) * accounts/abi: fix staticcheck warnings (ethereum#20358) * accounts/abi: fix staticcheck warnings * accounts/abi: restore unused field for test * core/state: fix staticcheck warnings (ethereum#20357) Also remove dependency on gopkg.in/check.v1 in tests. * metrics: fix issues reported by staticcheck (ethereum#20365) * trie: remove unused code (ethereum#20366) * p2p/discv5: add deprecation warning and remove unused code (ethereum#20367) * p2p/discv5: add deprecation warning and remove unused code * p2p/discv5: remove unused variables * tests, signer: remove staticcheck warnings (ethereum#20364) * core/asm: allow numbers in labels (ethereum#20362) Numbers were already allowed when creating labels, just not when referencing them. * miner: fix staticcheck warnings (ethereum#20375) * eth/tracers: fix staticcheck warnings (ethereum#20379) * trie: replace bigcache with fastcache (ethereum#19971) * cmd/geth: remove network id from version cmd It was reflective only of the Default setting, and not chain aware. * rlp: fix staticcheck warnings (ethereum#20368) * rlp: fix staticcheck warnings * rlp: fix ExampleDecode test * accounts/abi/bind: avoid reclaring structs (ethereum#20381) * accounts/keystore: fix staticcheck warnings (ethereum#20373) * accounts/keystore: fix staticcheck warnings * review feedback * p2p/discover: slow down lookups on empty table (ethereum#20389) * p2p/discover: slow down lookups on empty table * p2p/discover: wake from slowdown sleep when table is closed * les: fix clientInfo deadlock (ethereum#20395) * params: release go-ethereum v1.9.8 * params: begin v1.9.9 release cycle * cmd: fix command help messages in modules (ethereum#20203) * les: fix staticcheck warnings (ethereum#20371) * core: fix staticcheck warnings (ethereum#20384) * core: fix staticcheck warnings * fix goimports * core/rawdb: fix reinit regression caused by the hash check PR * deps: update fastcache to 1.5.3 * consensus/ethash: refactor remote sealer (ethereum#20335) The original idea behind this change was to remove a use of the deprecated CancelRequest method. Simply removing it would've been an option, but I couldn't resist and did a bit of a refactoring instead. All remote sealing code was contained in a single giant function. Remote sealing is now extracted into its own object, remoteSealer. * log: fix staticcheck warnings (ethereum#20388) * trie: remove dead code (ethereum#20405) * cmd/faucet, cmd/geth: fix staticcheck warnings (ethereum#20374) * internal: fix staticcheck warnings (ethereum#20380) * accounts/scwallet: fix staticcheck warnings (ethereum#20370) * internal/web3ext: add debug_accountRange (ethereum#20410) * accounts/usbwallet: fix staticcheck warnings (ethereum#20372) * core, miner: remove PostChainEvents (ethereum#19396) This change: - removes the PostChainEvents method on core.BlockChain. - sorts 'removed log' events by block number. - fire the NewChainHead event if we inject a canonical block into the chain even if the entire insertion is not successful. - guarantees correct event ordering in all cases. * accounts/abi/bind: fix destructive packing of *big.Int (ethereum#20412) * trie: track dirty cache metrics, track clean writes on commit * params: update CHTs for v1.9.9 release * p2p/enode: remove data race in sliceIter (ethereum#20421) * consensus/ethash, params: eip-2384: bump difficulty bomb (ethereum#20347) * consensus/ethash, params: implement eip-2384: bump difficulty bomb * params: EIP 2384 compat checks * consensus, params: add Muir Glacier block number (mainnet,ropsten) + official name * core/forkid: forkid tests for muir glacier * params/config: address review concerns * params, core/forkid: review nitpicks * cmd/geth,eth,les: add override option for muir glacier * params: nit fix * params: release Geth v1.9.9 * whisper/mailserver: reduce the max number of opened files (ethereum#18142) This should reduce the occurences of travis failures on MacOS Also fix some linter warnings * update default data path * use Constantinople for test net, from around December 30, 2019 * change default listening port to 30332 * set Constantinople Block to 3075000, which comes around January 15, 2020 at 11:19:06 am * fix data dir * Remove eth deployed checkpoint contents (#13) Co-authored-by: Péter Szilágyi <peterke@gmail.com> Co-authored-by: HackyMiner <hackyminer@gmail.com> Co-authored-by: lmittmann <lmittmann@users.noreply.github.com> Co-authored-by: Andrey Petrov <andrey.petrov@shazow.net> Co-authored-by: SjonHortensius <SjonHortensius@users.noreply.github.com> Co-authored-by: alexwang <39109351+dipingxian2@users.noreply.github.com> Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: Martin Holst Swende <martin@swende.se> Co-authored-by: Flash Sheridan <flash@pobox.com> Co-authored-by: gary rong <garyrong0905@gmail.com> Co-authored-by: winsvega <winsvega@mail.ru> Co-authored-by: Nguyen Kien Trung <trung.n.k@gmail.com> Co-authored-by: Rob Mulholand <rmulholand@8thlight.com> Co-authored-by: Felföldi Zsolt <zsfelfoldi@gmail.com> Co-authored-by: soc1c <soc1c@users.noreply.github.com> Co-authored-by: Rafael Matias <rafael@skyle.net> Co-authored-by: Lucas Hendren <lhendre2@gmail.com> Co-authored-by: Ryan Schneider <ryanleeschneider@gmail.com> Co-authored-by: ywzqwwt <39263032+ywzqwwt@users.noreply.github.com> Co-authored-by: zcheng9 <zcheng9@hawk.iit.edu> Co-authored-by: kikilass <36239971+kikilass@users.noreply.github.com> Co-authored-by: Darrel Herbst <dherbst@gmail.com> Co-authored-by: Ross <9055337+Chadsr@users.noreply.github.com> Co-authored-by: Jeffery Robert Walsh <rlxrlps@gmail.com> Co-authored-by: Marius Kjærstad <sandakersmann@users.noreply.github.com> Co-authored-by: Piotr Dyraga <piotr.dyraga@keep.network> Co-authored-by: Guillaume Ballet <gballet@gmail.com> Co-authored-by: Michael Forney <mforney@mforney.org> Co-authored-by: Samuel Marks <807580+SamuelMarks@users.noreply.github.com> Co-authored-by: Rick <rick.no@groundx.xyz> Co-authored-by: Kurkó Mihály <kurkomisi@users.noreply.github.com> Co-authored-by: Jorropo <jorropo.pgm@gmail.com> Co-authored-by: meowsbits <45600330+meowsbits@users.noreply.github.com> Co-authored-by: nebojsa94 <nebojsa94@users.noreply.github.com> Co-authored-by: meows <b5c6@protonmail.com> Co-authored-by: zaccoding <zaccoding725@gmail.com> Co-authored-by: xinluyin <31590468+xinluyin@users.noreply.github.com> Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de> Co-authored-by: neo lin <nlin@nbai.io>

holiman requested review from karalabe and rjl493456442 as code owners October 17, 2019 08:12

holiman force-pushed the less_memcpy branch from 49aaec0 to 9744064 Compare October 17, 2019 09:42

karalabe reviewed Oct 17, 2019

View reviewed changes

core/vm/memory.go Outdated Show resolved Hide resolved

karalabe reviewed Oct 17, 2019

View reviewed changes

core/evm, contracts: avoid copying memory for input in calls + make e…

8d39aed

…crecover not modify input buffer

holiman force-pushed the less_memcpy branch from 9744064 to 8d39aed Compare October 18, 2019 11:55

holiman changed the title ~~[wip] core/evm: avoid copying memory for input in calls~~ core/evm: avoid copying memory for input in calls Oct 23, 2019

core/vm: optimize mstore a bit

c9ff3e1

fjl requested changes Oct 28, 2019

View reviewed changes

core/vm: change Get -> GetCopy in vm memory access

33b3c05

karalabe reviewed Oct 30, 2019

View reviewed changes

karalabe approved these changes Oct 30, 2019

View reviewed changes

karalabe added this to the 1.9.7 milestone Nov 4, 2019

karalabe merged commit b566cfd into ethereum:master Nov 4, 2019

ehnuje mentioned this pull request Nov 25, 2020

blockchain/vm: avoid copying memory for input in calls klaytn/klaytn#780

Merged

9 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

core/evm: avoid copying memory for input in calls #20177

core/evm: avoid copying memory for input in calls #20177

holiman commented Oct 17, 2019 •

edited

Loading

holiman commented Oct 17, 2019

holiman commented Oct 17, 2019 •

edited

Loading

holiman commented Oct 17, 2019

karalabe commented Oct 17, 2019

karalabe Oct 17, 2019

holiman Oct 18, 2019

holiman Oct 23, 2019

holiman commented Oct 23, 2019

holiman commented Oct 24, 2019

fjl left a comment

holiman commented Oct 30, 2019

holiman commented Oct 30, 2019

karalabe Oct 30, 2019

holiman Oct 30, 2019

karalabe left a comment

core/evm: avoid copying memory for input in calls #20177

core/evm: avoid copying memory for input in calls #20177

Conversation

holiman commented Oct 17, 2019 • edited Loading

holiman commented Oct 17, 2019

holiman commented Oct 17, 2019 • edited Loading

holiman commented Oct 17, 2019

karalabe commented Oct 17, 2019

karalabe Oct 17, 2019

Choose a reason for hiding this comment

holiman Oct 18, 2019

Choose a reason for hiding this comment

holiman Oct 23, 2019

Choose a reason for hiding this comment

holiman commented Oct 23, 2019

holiman commented Oct 24, 2019

fjl left a comment

Choose a reason for hiding this comment

holiman commented Oct 30, 2019

holiman commented Oct 30, 2019

karalabe Oct 30, 2019

Choose a reason for hiding this comment

holiman Oct 30, 2019

Choose a reason for hiding this comment

karalabe left a comment

Choose a reason for hiding this comment

holiman commented Oct 17, 2019 •

edited

Loading

holiman commented Oct 17, 2019 •

edited

Loading