Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] EIP-7594: PeerDAS protocol #3574

Merged
merged 29 commits into from
Apr 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
d6a37ec
Copied from Danny's ethresearch post
hwwhww Nov 9, 2023
93dddd1
wip
hwwhww Nov 14, 2023
504b4f9
Migrating to latest crypto APIs
hwwhww Jan 11, 2024
696d443
Merge branch 'dev' into peer-das
hwwhww Jan 19, 2024
2cc7c87
Fix conflict
hwwhww Jan 19, 2024
665e6fa
Add `RowIndex`, `ColumnIndex` custom types in crypto doc
hwwhww Jan 19, 2024
9553d54
fix typo
hwwhww Jan 19, 2024
a72ece8
Apply suggestions from code review
hwwhww Jan 19, 2024
65be5b0
Make `CUSTODY_REQUIREMENT` unit be subnets; move some depended helper…
hwwhww Jan 19, 2024
55db861
Apply suggestions from code review
hwwhww Jan 20, 2024
4477cc6
Fix column computation
hwwhww Jan 20, 2024
56e6a98
`verify_data_column_sidecar_kzg_proof` -> `verify_data_column_sidecar…
hwwhww Jan 20, 2024
edeef07
toc
hwwhww Jan 28, 2024
b2a4657
Merge branch 'peer-das-req-subnet-count' into peer-das
hwwhww Jan 29, 2024
7aab577
Merge branch 'dev' into peer-das
hwwhww Jan 29, 2024
170dae5
Apply suggestions from code review
hwwhww Jan 29, 2024
547460c
Apply PR feedback
hwwhww Jan 30, 2024
d23452d
Deprecate `blob_sidecar_{subnet_id}`
hwwhww Jan 31, 2024
87e9702
Fix `DataColumnSidecarsByRoot`
hwwhww Jan 31, 2024
428c166
Apply suggestions from code review
hwwhww Feb 1, 2024
c47d5f3
Add `recover_matrix` and remove unused `FlatExtendedMatrix` type
hwwhww Feb 1, 2024
91dbbb3
Implement `compute_extended_matrix`
hwwhww Feb 1, 2024
e7c0d5f
Update specs/_features/eip7594/das-core.md
hwwhww Feb 2, 2024
8150f76
Apply @cskiraly's suggestion
hwwhww Feb 20, 2024
bb33f90
Change List length of `DataColumn` from `MAX_BLOBS_PER_BLOCK` to `MAX…
hwwhww Feb 20, 2024
1acb1ff
minor arrange
hwwhww Feb 20, 2024
cebf78a
Apply PR feedback
hwwhww Feb 27, 2024
8728561
Merge branch 'dev' into peer-das
hwwhww Apr 4, 2024
5535e6a
fix conflict
hwwhww Apr 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions configs/mainnet.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,9 @@ EIP7549_FORK_EPOCH: 18446744073709551615
# WHISK
WHISK_FORK_VERSION: 0x08000000 # temporary stub
WHISK_FORK_EPOCH: 18446744073709551615
# EIP7594
EIP7594_FORK_VERSION: 0x06000001
EIP7594_FORK_EPOCH: 18446744073709551615


# Time parameters
Expand Down Expand Up @@ -165,10 +168,10 @@ WHISK_EPOCHS_PER_SHUFFLING_PHASE: 256
WHISK_PROPOSER_SELECTION_GAP: 2

# EIP7594
EIP7594_FORK_VERSION: 0x06000001
EIP7594_FORK_EPOCH: 18446744073709551615
NUMBER_OF_COLUMNS: 128
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this is derived from other preset values, would it make more sense to keep it in the preset? (i.e. modifying this NUMBER_OF_COLUMNS in config would require a corresponding change on the FIELD_ELEMENTS_PER_BLOB/FIELD_ELEMENTS_PER_CELL values?

# `uint64((FIELD_ELEMENTS_PER_BLOB * 2) // FIELD_ELEMENTS_PER_CELL)` (= 128)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if it should be a config value, but I think it can be. I think it's okay for the config values to be derived from the preset values, but not the other way around. Preset values shouldn't be derived from config values.

DATA_COLUMN_SIDECAR_SUBNET_COUNT: 32
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For initial parametrization, I'd default to these values being equal. I'd only deviate from that len(columns) == len(subnets) if simulation/analysis shows we need to to support whatever we are targeting.

MAX_REQUEST_DATA_COLUMN_SIDECARS: 16384

# [New in EIP7251]
MIN_PER_EPOCH_CHURN_LIMIT_EIP7251: 128000000000 # 2**7 * 10**9 (= 128,000,000,000)
MAX_PER_EPOCH_ACTIVATION_EXIT_CHURN_LIMIT: 256000000000 # 2**8 * 10**9 (= 256,000,000,000)

9 changes: 6 additions & 3 deletions configs/minimal.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,9 @@ EIP7549_FORK_EPOCH: 18446744073709551615
# WHISK
WHISK_FORK_VERSION: 0x08000001
WHISK_FORK_EPOCH: 18446744073709551615

# EIP7594
EIP7594_FORK_VERSION: 0x06000001
EIP7594_FORK_EPOCH: 18446744073709551615

# Time parameters
# ---------------------------------------------------------------
Expand Down Expand Up @@ -164,8 +166,9 @@ WHISK_EPOCHS_PER_SHUFFLING_PHASE: 4
WHISK_PROPOSER_SELECTION_GAP: 1

# EIP7594
EIP7594_FORK_VERSION: 0x06000001
EIP7594_FORK_EPOCH: 18446744073709551615
NUMBER_OF_COLUMNS: 128
DATA_COLUMN_SIDECAR_SUBNET_COUNT: 32
MAX_REQUEST_DATA_COLUMN_SIDECARS: 16384

# [New in EIP7251]
MIN_PER_EPOCH_CHURN_LIMIT_EIP7251: 64000000000 # 2**6 * 10**9 (= 64,000,000,000)
Expand Down
2 changes: 2 additions & 0 deletions presets/mainnet/eip7594.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,5 @@
FIELD_ELEMENTS_PER_CELL: 64
# `uint64(2 * 4096)` (= 8192)
FIELD_ELEMENTS_PER_EXT_BLOB: 8192
# uint64(floorlog2(get_generalized_index(BeaconBlockBody, 'blob_kzg_commitments'))
KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH: 4
2 changes: 2 additions & 0 deletions presets/minimal/eip7594.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,5 @@
FIELD_ELEMENTS_PER_CELL: 64
# `uint64(2 * 4096)` (= 8192)
FIELD_ELEMENTS_PER_EXT_BLOB: 8192
# uint64(floorlog2(get_generalized_index(BeaconBlockBody, 'blob_kzg_commitments'))
KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH: 4
5 changes: 2 additions & 3 deletions pysetup/helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -208,8 +208,7 @@ def dependency_order_class_objects(objects: Dict[str, str], custom_types: Dict[s
for item in [dep, key] + key_list[key_list.index(dep)+1:]:
objects[item] = objects.pop(item)


def combine_ssz_objects(old_objects: Dict[str, str], new_objects: Dict[str, str], custom_types) -> Dict[str, str]:
def combine_ssz_objects(old_objects: Dict[str, str], new_objects: Dict[str, str]) -> Dict[str, str]:
"""
Takes in old spec and new spec ssz objects, combines them,
and returns the newer versions of the objects in dependency order.
Expand All @@ -231,7 +230,7 @@ def combine_spec_objects(spec0: SpecObject, spec1: SpecObject) -> SpecObject:
config_vars = combine_dicts(spec0.config_vars, spec1.config_vars)
ssz_dep_constants = combine_dicts(spec0.ssz_dep_constants, spec1.ssz_dep_constants)
func_dep_presets = combine_dicts(spec0.func_dep_presets, spec1.func_dep_presets)
ssz_objects = combine_ssz_objects(spec0.ssz_objects, spec1.ssz_objects, custom_types)
ssz_objects = combine_ssz_objects(spec0.ssz_objects, spec1.ssz_objects)
dataclasses = combine_dicts(spec0.dataclasses, spec1.dataclasses)
return SpecObject(
functions=functions,
Expand Down
17 changes: 16 additions & 1 deletion pysetup/md_doc_paths.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@
BELLATRIX: "sync/optimistic.md"
}

DEFAULT_ORDER = (
"beacon-chain",
"polynomial-commitments",
)


def is_post_fork(a, b) -> bool:
"""
Expand Down Expand Up @@ -66,15 +71,25 @@ def get_fork_directory(fork):
raise FileNotFoundError(f"No directory found for fork: {fork}")


def sort_key(s):
for index, key in enumerate(DEFAULT_ORDER):
if key in s:
return (index, s)
return (len(DEFAULT_ORDER), s)


def get_md_doc_paths(spec_fork: str) -> str:
md_doc_paths = ""

for fork in ALL_FORKS:
if is_post_fork(spec_fork, fork):
# Append all files in fork directory recursively
for root, dirs, files in os.walk(get_fork_directory(fork)):
for root, _, files in os.walk(get_fork_directory(fork)):
filepaths = []
for filename in files:
filepath = os.path.join(root, filename)
filepaths.append(filepath)
for filepath in sorted(filepaths, key=sort_key):
if filepath.endswith('.md') and filepath not in IGNORE_SPEC_FILES:
md_doc_paths += filepath + "\n"
# Append extra files if any
Expand Down
9 changes: 8 additions & 1 deletion pysetup/spec_builders/eip7594.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,15 @@ def imports(cls, preset_name: str):
'''

@classmethod
def hardcoded_custom_type_dep_constants(cls, spec_object) -> Dict[str, str]:
def hardcoded_custom_type_dep_constants(cls, spec_object) -> str:
return {
'FIELD_ELEMENTS_PER_CELL': spec_object.preset_vars['FIELD_ELEMENTS_PER_CELL'].value,
'FIELD_ELEMENTS_PER_EXT_BLOB': spec_object.preset_vars['FIELD_ELEMENTS_PER_EXT_BLOB'].value,
'NUMBER_OF_COLUMNS': spec_object.config_vars['NUMBER_OF_COLUMNS'].value,
}

@classmethod
def hardcoded_func_dep_presets(cls, spec_object) -> Dict[str, str]:
return {
'KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH': spec_object.preset_vars['KZG_COMMITMENTS_INCLUSION_PROOF_DEPTH'].value,
}
Loading
Loading