Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Block library refactoring #883

Merged
merged 29 commits into from
Oct 12, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c053761
block -> refactor: reworked header class with static factory instanti…
holgerd77 Sep 21, 2020
ff46c55
block -> refactoring: added new static factory helpers to block class
holgerd77 Sep 21, 2020
dd228bc
block -> refactor: fix build errors, remove unused imports, unpad num…
jochem-brouwer Sep 24, 2020
608d804
block -> rename Header to BlockHeader
jochem-brouwer Sep 28, 2020
17a33a8
block/tx -> fix block tests
jochem-brouwer Sep 28, 2020
72b5b78
block -> enforce BNs on fields which are interpreted as numbers
jochem-brouwer Sep 28, 2020
8adcdf9
block -> edge case in toBN
jochem-brouwer Sep 29, 2020
8987003
ethash -> make ethash compatible with block
jochem-brouwer Sep 29, 2020
5f5e9f2
Merge branch 'master' into refactor-block-library
ryanio Oct 7, 2020
5351fb8
have validateTransactions return a string[] (https://github.com/ether…
ryanio Oct 7, 2020
ce1dac1
let => const
ryanio Oct 7, 2020
526f986
set default param to resolve js runtime check
ryanio Oct 7, 2020
75689e6
continue refactoring and simplifying methods
ryanio Oct 8, 2020
8923f71
api updates
ryanio Oct 8, 2020
6a2c193
continuing work
ryanio Oct 8, 2020
381f5e1
inline buffer validations. add checks for extraData, mixHash and nonce
ryanio Oct 8, 2020
7eecf80
various fixups
ryanio Oct 8, 2020
395c6f8
continuing various work
ryanio Oct 9, 2020
be5c8d2
continuing work and refactoring
ryanio Oct 9, 2020
7fa486d
Merge branch 'master' into refactor-block-library
ryanio Oct 9, 2020
91d45d7
re-add timestamp to genesis (for rinkeby)
ryanio Oct 9, 2020
7e788c3
last fixups
ryanio Oct 9, 2020
bc459e8
update readme, benchmarks
ryanio Oct 9, 2020
ea8a401
update vm readme, simplify validate
ryanio Oct 10, 2020
b694010
fix timestamp validation
ryanio Oct 10, 2020
1f66378
use native eq
ryanio Oct 10, 2020
9f9bab0
make blockchain optional in block.validate()
ryanio Oct 10, 2020
7ce9132
fixups
ryanio Oct 10, 2020
a5d3d14
remove BLOCK_difficulty_GivenAsList from skip list (https://github.co…
ryanio Oct 12, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11,903 changes: 10,092 additions & 1,811 deletions package-lock.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion packages/account/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
},
"homepage": "https://github.com/ethereumjs/ethereumjs-vm/tree/master/packages/account#synopsis",
"dependencies": {
"ethereumjs-util": "^7.0.5",
"ethereumjs-util": "^7.0.6",
"rlp": "^2.2.3",
"safe-buffer": "^5.1.1"
},
Expand Down
2 changes: 1 addition & 1 deletion packages/block/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
"@ethereumjs/common": "^1.5.1",
"@ethereumjs/tx": "^2.1.2",
"@types/bn.js": "^4.11.6",
"ethereumjs-util": "^7.0.5",
"ethereumjs-util": "^7.0.6",
"merkle-patricia-tree": "^4.0.0"
},
"devDependencies": {
Expand Down
262 changes: 136 additions & 126 deletions packages/block/src/block.ts
Original file line number Diff line number Diff line change
@@ -1,200 +1,222 @@
import { BaseTrie as Trie } from 'merkle-patricia-tree'
import { BN, rlp, keccak256, KECCAK256_RLP, baToJSON } from 'ethereumjs-util'
import { rlp, keccak256, KECCAK256_RLP } from 'ethereumjs-util'
import Common from '@ethereumjs/common'
import { Transaction } from '@ethereumjs/tx'
import { Transaction, TxOptions } from '@ethereumjs/tx'
import { BlockHeader } from './header'
import { Blockchain, BlockData, BlockOptions } from './types'
import { BlockData, BlockOptions, JsonBlock, BlockBuffer, Blockchain } from './types'

/**
* An object that represents the block
* An object that represents the block.
*/
export class Block {
public readonly header: BlockHeader
public readonly transactions: Transaction[] = []
public readonly uncleHeaders: BlockHeader[] = []
public readonly txTrie = new Trie()

public readonly _common: Common

/**
* Creates a new block object
*
* Please solely use this constructor to pass in block header data
* and don't modfiy header data after initialization since this can lead to
* undefined behavior regarding HF rule implemenations within the class.
*
* @param data - The block's data.
* @param options - The options for this block (like the chain setup)
*/
constructor(
data: Buffer | [Buffer[], Buffer[], Buffer[]] | BlockData = {},
options: BlockOptions = {},
) {
// Checking at runtime, to prevent errors down the path for JavaScript consumers.
if (data === null) {
data = {}
}
public static fromBlockData(blockData: BlockData = {}, opts: BlockOptions = {}) {
const { header: headerData, transactions: txsData, uncleHeaders: uhsData } = blockData

let rawTransactions
let rawUncleHeaders
const header = BlockHeader.fromHeaderData(headerData, opts)

if (Buffer.isBuffer(data)) {
// We do this to silence a TS error. We know that after this statement, data is
// a [Buffer[], Buffer[], Buffer[]]
const dataAsAny = rlp.decode(data) as any
data = dataAsAny as [Buffer[], Buffer[], Buffer[]]
// parse transactions
const transactions = []
for (const txData of txsData || []) {
const tx = Transaction.fromTxData(txData, opts as TxOptions)
transactions.push(tx)
}

// Initialize the block header
if (Array.isArray(data)) {
this.header = new BlockHeader(data[0], options)
rawTransactions = data[1]
rawUncleHeaders = data[2]
} else {
this.header = new BlockHeader(data.header, options)
rawTransactions = data.transactions || []
rawUncleHeaders = data.uncleHeaders || []
// parse uncle headers
const uncleHeaders = []
for (const uhData of uhsData || []) {
const uh = BlockHeader.fromHeaderData(uhData, opts)
uncleHeaders.push(uh)
}
this._common = this.header._common

// parse uncle headers
for (let i = 0; i < rawUncleHeaders.length; i++) {
this.uncleHeaders.push(new BlockHeader(rawUncleHeaders[i], options))
return new Block(header, transactions, uncleHeaders)
}

public static fromRLPSerializedBlock(serialized: Buffer, opts: BlockOptions = {}) {
const values = (rlp.decode(serialized) as any) as BlockBuffer

if (!Array.isArray(values)) {
throw new Error('Invalid serialized block input. Must be array')
}

return Block.fromValuesArray(values, opts)
}

public static fromValuesArray(values: BlockBuffer, opts: BlockOptions = {}) {
if (values.length > 3) {
throw new Error('invalid block. More values than expected were received')
}

const [headerData, txsData, uhsData] = values

const header = BlockHeader.fromValuesArray(headerData, opts)

// parse transactions
const txOpts = { common: this._common }
for (let i = 0; i < rawTransactions.length; i++) {
const txData = rawTransactions[i]
const tx = Array.isArray(txData)
? Transaction.fromValuesArray(txData as Buffer[], txOpts)
: Transaction.fromRlpSerializedTx(txData as Buffer, txOpts)
this.transactions.push(tx)
const transactions = []
for (const txData of txsData || []) {
transactions.push(Transaction.fromValuesArray(txData, opts))
}

// parse uncle headers
const uncleHeaders = []
for (const uncleHeaderData of uhsData || []) {
uncleHeaders.push(BlockHeader.fromValuesArray(uncleHeaderData, opts))
}

return new Block(header, transactions, uncleHeaders)
}

/**
* Alias for Block.fromBlockData() with initWithGenesisHeader set to true.
*/
public static genesis(blockData: BlockData = {}, opts: BlockOptions = {}) {
opts = { ...opts, initWithGenesisHeader: true }
return Block.fromBlockData(blockData, opts)
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thought we can really keep this as simple as you proposed (or more or less), so optimally Block.genesis(common), and everyone needing something adjusted can just use the Block.fromBlockData() variant?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I wanted to too but in practice (in our test suite) it was helpful to accept custom overrides as well.


/**
* This constructor takes the values, validates them, assigns them and freezes the object.
* Use the static factory methods to assist in creating a Block object from varying data types and options.
*/
constructor(
header?: BlockHeader,
transactions: Transaction[] = [],
uncleHeaders: BlockHeader[] = [],
opts: BlockOptions = {},
) {
this.header = header || BlockHeader.fromHeaderData({}, opts)
this.transactions = transactions
this.uncleHeaders = uncleHeaders
this._common = this.header._common

Object.freeze(this)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For understanding on further implications: does this Object.freeze() mean we can't do further modifications on e.g. the Common instance passed?

So would a subsequent call (e.g. in a VM context) common.setHardfork('byzantium') (or whatever) throw in this context or would this still work?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll have to give it a test, but I think the common object can still update, it just can't be set to a totally new instance (e.g. block._common = newCommon would fail)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think this is the case according to here:

Note that values that are objects can still be modified, unless they are also frozen.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/freeze

}

get raw(): [Buffer[], Buffer[], Buffer[]] {
return this.serialize(false)
/**
* Returns a Buffer Array of the raw Buffers of this block, in order.
*/
raw(): BlockBuffer {
return [
this.header.raw(),
this.transactions.map((tx) => tx.raw()),
this.uncleHeaders.map((uh) => uh.raw()),
]
}

/**
* Produces a hash the RLP of the block
* Produces a hash the RLP of the block.
*/
hash(): Buffer {
return this.header.hash()
}

/**
* Determines if this block is the genesis block
* Determines if this block is the genesis block.
*/
isGenesis(): boolean {
return this.header.isGenesis()
}

/**
* Produces a serialization of the block.
*
* @param rlpEncode - If `true`, the returned object is the RLP encoded data as seen by the
* Ethereum wire protocol. If `false`, a tuple with the raw data of the header, the txs and the
* uncle headers is returned.
* Returns the rlp encoding of the block.
*/
serialize(): Buffer
serialize(rlpEncode: true): Buffer
serialize(rlpEncode: false): [Buffer[], Buffer[], Buffer[]]
serialize(rlpEncode = true) {
const raw = [
this.header.raw,
this.transactions.map((tx) => tx.serialize()),
this.uncleHeaders.map((uh) => uh.raw),
]

return rlpEncode ? rlp.encode(raw) : raw
serialize(): Buffer {
return rlp.encode(this.raw())
}

/**
* Generate transaction trie. The tx trie must be generated before the transaction trie can
* be validated with `validateTransactionTrie`
* Generates transaction trie for validation.
*/
async genTxTrie(): Promise<void> {
for (let i = 0; i < this.transactions.length; i++) {
const tx = this.transactions[i]
await this._putTxInTrie(i, tx)
const { transactions, txTrie } = this
for (let i = 0; i < transactions.length; i++) {
const tx = transactions[i]
const key = rlp.encode(i)
const value = tx.serialize()
await txTrie.put(key, value)
}
}

/**
* Validates the transaction trie
* Validates the transaction trie.
*/
validateTransactionsTrie(): boolean {
if (this.transactions.length) {
return this.header.transactionsTrie.equals(this.txTrie.root)
} else {
async validateTransactionsTrie(): Promise<boolean> {
if (this.transactions.length === 0) {
return this.header.transactionsTrie.equals(KECCAK256_RLP)
}

if (this.txTrie.root.equals(KECCAK256_RLP)) {
await this.genTxTrie()
}

return this.txTrie.root.equals(this.header.transactionsTrie)
}

/**
* Validates the transactions
* Validates the transactions.
*
* @param stringError - If `true`, a string with the indices of the invalid txs is returned.
*/
validateTransactions(): boolean
validateTransactions(stringError: false): boolean
validateTransactions(stringError: true): string
validateTransactions(stringError: true): string[]
validateTransactions(stringError = false) {
const errors: string[] = []

this.transactions.forEach(function (tx, i) {
const errs = tx.validate(true)
if (errs.length !== 0) {
if (errs.length > 0) {
errors.push(`errors at tx ${i}: ${errs.join(', ')}`)
}
})

return stringError ? errors.join(' ') : errors.length === 0
return stringError ? errors : errors.length === 0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, just re-read the discussion with @alcuadrado, this really makes a lot of sense to skip the join here.

}

/**
* Validates the entire block, throwing if invalid.
* Validates the block, throwing if invalid.
*
* @param blockchain - the blockchain that this block wants to be part of
* @param blockchain - additionally validate against a @ethereumjs/blockchain
*/
async validate(blockchain: Blockchain): Promise<void> {
await Promise.all([
this.validateUncles(blockchain),
this.genTxTrie(),
this.header.validate(blockchain),
])

if (!this.validateTransactionsTrie()) {
throw new Error('invalid transaction trie')
}
async validate(blockchain?: Blockchain): Promise<void> {
await this.header.validate(blockchain)

const txErrors = this.validateTransactions(true)
if (txErrors !== '') {
throw new Error(txErrors)
if (txErrors.length > 0) {
throw new Error(`invalid transactions: ${txErrors.join(' ')}`)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

}

const validateTxTrie = await this.validateTransactionsTrie()
if (!validateTxTrie) {
throw new Error('invalid transaction trie')
}

await this.validateUncles(blockchain)

if (!this.validateUnclesHash()) {
throw new Error('invalid uncle hash')
}
}

/**
* Validates the uncle's hash
* Validates the uncle's hash.
*/
validateUnclesHash(): boolean {
const raw = rlp.encode(this.uncleHeaders.map((uh) => uh.raw))

const raw = rlp.encode(this.uncleHeaders.map((uh) => uh.raw()))
return keccak256(raw).equals(this.header.uncleHash)
}

/**
* Validates the uncles that are in the block, if any. This method throws if they are invalid.
*
* @param blockchain - the blockchain that this block wants to be part of
* @param blockchain - additionally validate against a @ethereumjs/blockchain
*/
async validateUncles(blockchain: Blockchain): Promise<void> {
async validateUncles(blockchain?: Blockchain): Promise<void> {
if (this.isGenesis()) {
return
}
Expand All @@ -204,43 +226,31 @@ export class Block {
}

const uncleHashes = this.uncleHeaders.map((header) => header.hash().toString('hex'))

if (!(new Set(uncleHashes).size === uncleHashes.length)) {
throw new Error('duplicate uncles')
}

await Promise.all(
this.uncleHeaders.map(async (uh) => this._validateUncleHeader(uh, blockchain)),
)
for (const uh of this.uncleHeaders) {
await this._validateUncleHeader(uh, blockchain)
}
}

/**
* Returns the block in JSON format
*
* @see {@link https://github.com/ethereumjs/ethereumjs-util/blob/master/docs/index.md#defineproperties|ethereumjs-util}
* Returns the block in JSON format.
*/
toJSON(labeled: boolean = false) {
if (labeled) {
return {
header: this.header.toJSON(true),
transactions: this.transactions.map((tx) => tx.toJSON()),
uncleHeaders: this.uncleHeaders.forEach((uh) => uh.toJSON(true)),
}
} else {
return baToJSON(this.raw)
toJSON(): JsonBlock {
return {
header: this.header.toJSON(),
transactions: this.transactions.map((tx) => tx.toJSON()),
uncleHeaders: this.uncleHeaders.map((uh) => uh.toJSON()),
}
}

private async _putTxInTrie(txIndex: number, tx: Transaction) {
await this.txTrie.put(rlp.encode(txIndex), tx.serialize())
}

private _validateUncleHeader(uncleHeader: BlockHeader, blockchain: Blockchain) {
private _validateUncleHeader(uncleHeader: BlockHeader, blockchain?: Blockchain) {
// TODO: Validate that the uncle header hasn't been included in the blockchain yet.
// This is not possible in ethereumjs-blockchain since this PR was merged:
// https://github.com/ethereumjs/ethereumjs-blockchain/pull/47

const height = new BN(this.header.number)
const height = this.header.number
return uncleHeader.validate(blockchain, height)
}
}
Loading