Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update SSZ spec to use SOS style offset based layout. #787

Merged
Merged
Changes from 16 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
ad07649
Update spec to use SOS style offset layout for variable size values.
pipermerriam Mar 18, 2019
4a0459a
PR feedback
pipermerriam Mar 18, 2019
605028b
more precise definitions for and and expand code example for how se…
pipermerriam Mar 18, 2019
fa66640
Update specs/simple-serialize.md
jannikluhn Mar 20, 2019
32684d5
Update specs/simple-serialize.md
jannikluhn Mar 20, 2019
3741b75
Update specs/simple-serialize.md
jannikluhn Mar 20, 2019
1ab5019
Update specs/simple-serialize.md
jannikluhn Mar 20, 2019
ca98d75
d
pipermerriam Mar 20, 2019
5f46584
more language updates
pipermerriam Mar 20, 2019
66173b8
static > fixed
pipermerriam Mar 20, 2019
92f002c
specify offsets better
pipermerriam Mar 28, 2019
4d2bdf8
Cleanup spec
JustinDrake Apr 13, 2019
aaa5a16
Update simple-serialize.md
JustinDrake Apr 13, 2019
0695d0a
Update simple-serialize.md
JustinDrake Apr 13, 2019
35a6311
Update simple-serialize.md
JustinDrake Apr 13, 2019
80bd4a3
Update simple-serialize.md
JustinDrake Apr 13, 2019
10f3db9
Update simple-serialize.md
JustinDrake Apr 13, 2019
27cf02a
Update simple-serialize.md
JustinDrake Apr 13, 2019
a90bcc0
Update simple-serialize.md
JustinDrake Apr 13, 2019
23c0954
Update simple-serialize.md
JustinDrake Apr 13, 2019
f6ed1df
Update simple-serialize.md
JustinDrake Apr 13, 2019
9adbaba
Update simple-serialize.md
JustinDrake Apr 13, 2019
97ca672
Update simple-serialize.md
JustinDrake Apr 13, 2019
09d9274
Update simple-serialize.md
JustinDrake Apr 13, 2019
7255b0f
Update simple-serialize.md
JustinDrake Apr 13, 2019
59f5680
Update simple-serialize.md
JustinDrake Apr 13, 2019
62ffb89
Update simple-serialize.md
JustinDrake Apr 13, 2019
1284b93
Update simple-serialize.md
JustinDrake Apr 24, 2019
7694b9e
Merge branch 'dev' into piper/add-sos-style-offset-serialization-to-ssz
djrtwo Apr 24, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 17 additions & 13 deletions specs/simple-serialize.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,9 @@ This is a **work in progress** describing typing, serialization and Merkleizatio

| Name | Value | Description |
|-|-|-|
| `BYTES_PER_CHUNK` | `32` | Number of bytes per chunk.
| `BYTES_PER_LENGTH_PREFIX` | `4` | Number of bytes per serialized length prefix. |
| `BYTES_PER_CHUNK` | `32` | Number of bytes per chunk. |
| `BYTES_PER_LENGTH_OFFSET` | `4` | Number of bytes per serialized length prefix. |
| `BITS_PER_BYTE` | `8` | Number of bits per byte. |

## Typing
### Basic types
Expand Down Expand Up @@ -54,7 +55,7 @@ For convenience we alias:

We recursively define the `serialize` function which consumes an object `value` (of the type specified) and returns a bytestring of type `"bytes"`.

*Note*: In the function definitions below (`serialize`, `hash_tree_root`, `signed_root`, etc.) objects implicitly carry their type.
> *Note*: In the function definitions below (`serialize`, `hash_tree_root`, `signed_root`, `is_fixed_size`, `is_variable_size`, etc.) objects implicitly carry their type.
JustinDrake marked this conversation as resolved.
Show resolved Hide resolved

### `"uintN"`

Expand All @@ -72,19 +73,22 @@ return b"\x01" if value is True else b"\x00"

### Vectors, containers, lists

If `value` is fixed-size:

```python
return "".join([serialize(element) for element in value])
```
# Reccursively serialize
fixed_parts = [serialize(element) if is_fixed_size(element) else None for element in value]
variable_parts = [serialize(element) if is_variable_size(element) else None for element in value]

If `value` is variable-size:
# Compute and check lengths
fixed_lengths = [len(part) if part != None else BYTES_PER_LENGTH_OFFSET for part in fixed_parts]
variable_lengths = [len(part) if part != None else 0 for part in variable_parts]
assert sum(fixed_lengths + variable_lengths) < 2**(BYTES_PER_LENGTH_OFFSET * BITS_PER_BYTE)

```python
serialized_bytes = "".join([serialize(element) for element in value])
assert len(serialized_bytes) < 2**(8 * BYTES_PER_LENGTH_PREFIX)
serialized_length = len(serialized_bytes).to_bytes(BYTES_PER_LENGTH_PREFIX, "little")
return serialized_length + serialized_bytes
# Compute offsets of variable-size elements, and interleave with fixed parts
offsets = [sum(fixed_lengths) + sum(variable_lengths[:i]) for i in range(len(value))]
fixed_parts = [part if part != None else offsets[i] for i, part in enumerate(fixed_parts)]

# Return the fixed parts (with offsets interleaved) followed by variable parts
return "".join(fixed_parts + variable_parts)
JustinDrake marked this conversation as resolved.
Show resolved Hide resolved
```

## Deserialization
Expand Down