Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement EIP4895: Beacon Chain withdrawals #2353

Merged
merged 9 commits into from
Nov 3, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 87 additions & 8 deletions packages/block/src/block.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,30 @@ import {
bufferToHex,
intToHex,
isHexPrefixed,
toBuffer,
} from '@ethereumjs/util'
import { keccak256 } from 'ethereum-cryptography/keccak'
import { ethers } from 'ethers'

import { blockFromRpc } from './from-rpc'
import { BlockHeader } from './header'

import type { BlockBuffer, BlockData, BlockOptions, JsonBlock, JsonRpcBlock } from './types'
import type {
BlockBuffer,
BlockData,
BlockOptions,
JsonBlock,
JsonRpcBlock,
Withdrawal,
} from './types'
import type { Common } from '@ethereumjs/common'
import type {
FeeMarketEIP1559Transaction,
Transaction,
TxOptions,
TypedTransaction,
} from '@ethereumjs/tx'
import type { Address } from '@ethereumjs/util'

/**
* An object that represents the block.
Expand All @@ -33,6 +42,7 @@ export class Block {
public readonly header: BlockHeader
public readonly transactions: TypedTransaction[] = []
public readonly uncleHeaders: BlockHeader[] = []
public readonly withdrawals?: Withdrawal[]
Copy link
Member

@holgerd77 holgerd77 Nov 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about this for some time now: withdrawals have this type Withdrawal here with all these non-deterministic types like BigIntLike:

export type Withdrawal = {
  index: BigIntLike
  validatorIndex: BigIntLike
  address: AddressLike
  amount: BigIntLike
}

While this always comes in handy for input data I wonder if this is an optimal design choice for the internal representation of this data. In the case of withdrawals, this Withdrawal type atm actually serves for both puroposes.

With this kind of design we just can't be sure what types these internal withdrawal parameters like index or address really have. My fear is that this will lead to a myriad of follow-up problems. For test cases, client usages,... one can never really count on what the type really is here and either cast here, make code comments, add complex procedural choice conditional switches for values on processing, or minimally: one always need to convert with withdrawalToBufferArray().

I would have a relatively strong tendency that we should switch here to deterministic values, I guess that will safe us from some substantial amount of follow-up trouble. This also aligns with our standard handling of internally represented data.

So I would suggest to actually have two types here, the one from above renamed to WithdrawalData or similar and used for withdrawal inputs.

And another one Withdrawal with deterministic types. Following our tendency to represent internal values not always with Buffer any more but with more "matching" types this would be something like the following I guess:

export type Withdrawal = {
  index: BigInt
  validatorIndex: BigInt
  address: Address
  amount: BigInt
}

I know that this is some extra effort to rewrite now with various code already in place but I am relatively sure that this more than pay off in the future.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: Haha https://github.com/ethereumjs/ethereumjs-monorepo/pull/2401/files#diff-6ef8827503f0272ab6d926170261d659f841b39a00e354abcd69ea5635c8eae1

Just discovered on the follow up PR from @g11tech.

At least we think in impressively similar lines. 🤩 😂

public readonly txTrie = new Trie()
public readonly _common: Common

Expand All @@ -43,7 +53,12 @@ export class Block {
* @param opts
*/
public static fromBlockData(blockData: BlockData = {}, opts?: BlockOptions) {
const { header: headerData, transactions: txsData, uncleHeaders: uhsData } = blockData
const {
header: headerData,
transactions: txsData,
uncleHeaders: uhsData,
withdrawals,
} = blockData
const header = BlockHeader.fromHeaderData(headerData, opts)

// parse transactions
Expand Down Expand Up @@ -78,7 +93,7 @@ export class Block {
uncleHeaders.push(uh)
}

return new Block(header, transactions, uncleHeaders, opts)
return new Block(header, transactions, uncleHeaders, opts, withdrawals)
}

/**
Expand All @@ -104,11 +119,11 @@ export class Block {
* @param opts
*/
public static fromValuesArray(values: BlockBuffer, opts?: BlockOptions) {
if (values.length > 3) {
if (values.length > 4) {
throw new Error('invalid block. More values than expected were received')
}

const [headerData, txsData, uhsData] = values
const [headerData, txsData, uhsData, withdrawalsData] = values

const header = BlockHeader.fromValuesArray(headerData, opts)

Expand Down Expand Up @@ -144,7 +159,16 @@ export class Block {
uncleHeaders.push(BlockHeader.fromValuesArray(uncleHeaderData, uncleOpts))
}

return new Block(header, transactions, uncleHeaders, opts)
let withdrawals
if (withdrawalsData) {
withdrawals = <Withdrawal[]>[]
for (const withdrawal of withdrawalsData) {
const [index, validatorIndex, address, amount] = withdrawal
withdrawals.push({ index, validatorIndex, address, amount })
}
}

return new Block(header, transactions, uncleHeaders, opts, withdrawals)
}

/**
Expand Down Expand Up @@ -212,7 +236,8 @@ export class Block {
header?: BlockHeader,
transactions: TypedTransaction[] = [],
uncleHeaders: BlockHeader[] = [],
opts: BlockOptions = {}
opts: BlockOptions = {},
withdrawals?: Withdrawal[]
) {
this.header = header ?? BlockHeader.fromHeaderData({}, opts)
this.transactions = transactions
Expand All @@ -234,23 +259,55 @@ export class Block {
}
}

if (this._common.isActivatedEIP(4895) && withdrawals === undefined) {
throw new Error('Need a withdrawals field if EIP 4895 is active')
} else if (!this._common.isActivatedEIP(4895) && withdrawals !== undefined) {
throw new Error('Cannot have a withdrawals field if EIP 4895 is not active')
}

this.withdrawals = withdrawals

const freeze = opts?.freeze ?? true
if (freeze) {
Object.freeze(this)
}
}

/**
* Convert a withdrawal to a buffer array
* @param withdrawal the withdrawal to convert
* @returns buffer array of the withdrawal
*/
private withdrawalToBufferArray(withdrawal: Withdrawal): [Buffer, Buffer, Buffer, Buffer] {
const { index, validatorIndex, address, amount } = withdrawal
let addressBuffer: Buffer
if (typeof address === 'string') {
addressBuffer = Buffer.from(address.slice(2))
} else if (Buffer.isBuffer(address)) {
addressBuffer = address
} else {
addressBuffer = (<Address>address).buf
}
return [toBuffer(index), toBuffer(validatorIndex), addressBuffer, toBuffer(amount)]
}

/**
* Returns a Buffer Array of the raw Buffers of this block, in order.
*/
raw(): BlockBuffer {
return [
const bufferArray = <BlockBuffer>[
this.header.raw(),
this.transactions.map((tx) =>
tx.supports(Capability.EIP2718TypedTransaction) ? tx.serialize() : tx.raw()
) as Buffer[],
this.uncleHeaders.map((uh) => uh.raw()),
]
if (this.withdrawals) {
for (const withdrawal of this.withdrawals) {
bufferArray.push(this.withdrawalToBufferArray(withdrawal))
}
}
return bufferArray
}

/**
Expand Down Expand Up @@ -369,6 +426,11 @@ export class Block {
const msg = this._errorMsg('invalid uncle hash')
throw new Error(msg)
}

if (this._common.isActivatedEIP(4895) && !(await this.validateWithdrawalsTrie())) {
const msg = this._errorMsg('invalid withdrawals trie')
throw new Error(msg)
}
}

/**
Expand All @@ -380,6 +442,23 @@ export class Block {
return Buffer.from(keccak256(raw)).equals(this.header.uncleHash)
}

/**
* Validates the withdrawal root
*/
async validateWithdrawalsTrie(): Promise<boolean> {
if (!this._common.isActivatedEIP(4895)) {
throw new Error('EIP 4895 is not activated')
}
const trie = new Trie()
let index = 0
for (const withdrawal of this.withdrawals!) {
const withdrawalRLP = RLP.encode(this.withdrawalToBufferArray(withdrawal))
await trie.put(Buffer.from('0x' + index.toString(16)), arrToBufArr(withdrawalRLP))
index++
}
return trie.root().equals(this.header.withdrawalsRoot!)
}

/**
* Consistency checks for uncles included in the block, if any.
*
Expand Down
30 changes: 30 additions & 0 deletions packages/block/src/header.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ export class BlockHeader {
public readonly mixHash: Buffer
public readonly nonce: Buffer
public readonly baseFeePerGas?: bigint
public readonly withdrawalsRoot?: Buffer

public readonly _common: Common

Expand Down Expand Up @@ -154,6 +155,7 @@ export class BlockHeader {
mixHash: zeros(32),
nonce: zeros(8),
baseFeePerGas: undefined,
withdrawalsRoot: undefined,
}

const parentHash = toType(headerData.parentHash, TypeOutput.Buffer) ?? defaults.parentHash
Expand All @@ -176,6 +178,8 @@ export class BlockHeader {
const nonce = toType(headerData.nonce, TypeOutput.Buffer) ?? defaults.nonce
let baseFeePerGas =
toType(headerData.baseFeePerGas, TypeOutput.BigInt) ?? defaults.baseFeePerGas
const withdrawalsRoot =
toType(headerData.withdrawalsRoot, TypeOutput.Buffer) ?? defaults.withdrawalsRoot

const hardforkByBlockNumber = options.hardforkByBlockNumber ?? false
if (hardforkByBlockNumber || options.hardforkByTTD !== undefined) {
Expand All @@ -198,6 +202,18 @@ export class BlockHeader {
}
}

if (this._common.isActivatedEIP(4895)) {
if (withdrawalsRoot === defaults.withdrawalsRoot) {
throw new Error('invalid header. withdrawalsRoot should be provided')
}
} else {
if (withdrawalsRoot !== undefined) {
throw new Error(
'A withdrawalsRoot for a header can only be provied with EIP4895 being activated'
)
}
}

this.parentHash = parentHash
this.uncleHash = uncleHash
this.coinbase = coinbase
Expand All @@ -214,6 +230,7 @@ export class BlockHeader {
this.mixHash = mixHash
this.nonce = nonce
this.baseFeePerGas = baseFeePerGas
this.withdrawalsRoot = withdrawalsRoot

this._genericFormatValidation()
this._validateDAOExtraData()
Expand Down Expand Up @@ -310,6 +327,19 @@ export class BlockHeader {
}
}
}

if (this._common.isActivatedEIP(4895) === true) {
if (this.withdrawalsRoot === undefined) {
const msg = this._errorMsg('EIP4895 block has no withdrawalsRoot field')
throw new Error(msg)
}
if (this.withdrawalsRoot?.length !== 32) {
const msg = this._errorMsg(
`withdrawalsRoot must be 32 bytes, received ${this.withdrawalsRoot!.length} bytes`
)
throw new Error(msg)
}
}
}

/**
Expand Down
4 changes: 3 additions & 1 deletion packages/block/src/helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,10 @@ export function valuesArrayToHeaderData(values: BlockHeaderBuffer): HeaderData {
mixHash,
nonce,
baseFeePerGas,
withdrawalsRoot,
] = values

if (values.length > 16) {
if (values.length > 17) {
throw new Error('invalid header. More values than expected were received')
}
if (values.length < 15) {
Expand All @@ -63,6 +64,7 @@ export function valuesArrayToHeaderData(values: BlockHeaderBuffer): HeaderData {
mixHash,
nonce,
baseFeePerGas,
withdrawalsRoot,
}
}

Expand Down
26 changes: 24 additions & 2 deletions packages/block/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,14 @@ export interface HeaderData {
mixHash?: BufferLike
nonce?: BufferLike
baseFeePerGas?: BigIntLike
withdrawalsRoot?: BufferLike
}

export type Withdrawal = {
index: BigIntLike
validatorIndex: BigIntLike
address: AddressLike
amount: BigIntLike
}

/**
Expand All @@ -107,16 +115,20 @@ export interface BlockData {
header?: HeaderData
transactions?: Array<TxData | AccessListEIP2930TxData | FeeMarketEIP1559TxData>
uncleHeaders?: Array<HeaderData>
withdrawals?: Array<Withdrawal>
}

export type BlockBuffer = [BlockHeaderBuffer, TransactionsBuffer, UncleHeadersBuffer]
export type BlockBuffer =
| [BlockHeaderBuffer, TransactionsBuffer, UncleHeadersBuffer]
| [BlockHeaderBuffer, TransactionsBuffer, UncleHeadersBuffer, WithdrawalBuffer]
export type BlockHeaderBuffer = Buffer[]
export type BlockBodyBuffer = [TransactionsBuffer, UncleHeadersBuffer]
export type BlockBodyBuffer = [TransactionsBuffer, UncleHeadersBuffer, WithdrawalBuffer?]
/**
* TransactionsBuffer can be an array of serialized txs for Typed Transactions or an array of Buffer Arrays for legacy transactions.
*/
export type TransactionsBuffer = Buffer[][] | Buffer[]
export type UncleHeadersBuffer = Buffer[][]
export type WithdrawalBuffer = Buffer[][]

/**
* An object with the block's data represented as strings.
Expand All @@ -128,6 +140,7 @@ export interface JsonBlock {
header?: JsonHeader
transactions?: JsonTx[]
uncleHeaders?: JsonHeader[]
withdrawals?: JsonRpcWithdrawal[]
}

/**
Expand All @@ -150,6 +163,14 @@ export interface JsonHeader {
mixHash?: string
nonce?: string
baseFeePerGas?: string
withdrawalsRoot?: string
}

export interface JsonRpcWithdrawal {
index: string // QUANTITY - bigint 8 bytes
validatorIndex: string // QUANTITY - bigint 8 bytes
address: string // DATA, 20 Bytes address to withdraw to
amount: string // QUANTITY - bigint amount in wei 32 bytes
}

/*
Expand Down Expand Up @@ -177,4 +198,5 @@ export interface JsonRpcBlock {
transactions: Array<JsonRpcTx | string> // Array of transaction objects, or 32 Bytes transaction hashes depending on the last given parameter.
uncles: string[] // Array of uncle hashes
baseFeePerGas?: string // If EIP-1559 is enabled for this block, returns the base fee per gas
withdrawals?: Array<JsonRpcWithdrawal> // If EIP-4895 is enabled for this block, array of withdrawals
}
Loading