Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Atree Register Inlining and Data Deduplication #342

Merged
merged 56 commits into from
Oct 4, 2023
Merged

Atree Register Inlining and Data Deduplication #342

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
d6f3daa
Inline child array/map data slab into parent slab
fxamacker Sep 14, 2023
6f25137
Check overwritten value in parentUpdater callback
fxamacker Sep 17, 2023
399684a
Avoid writing parent on update to uninlinable child
fxamacker Sep 18, 2023
c178a72
Update comments about encoding format
fxamacker Sep 19, 2023
efc46e6
Update comments about encoding format
fxamacker Sep 19, 2023
1664e36
Add notification in Set and Insert in Array
fxamacker Sep 20, 2023
feb6adf
Add notification in Set in OrderedMap
fxamacker Sep 20, 2023
0d57f80
Deduplicate composite by type ID and field names
fxamacker Sep 21, 2023
1643589
Add comment to update ValueID when SlabID is changed
fxamacker Sep 21, 2023
72d3614
Use new ExtraData for inlined slab during decoding
fxamacker Sep 21, 2023
f66a85b
Copy key for inlined composite during decoding
fxamacker Sep 21, 2023
f8e0992
Add comments for safe use of range loop over Go map
fxamacker Sep 21, 2023
3584952
Replace if with switch in Array.Storable()
fxamacker Sep 21, 2023
dc8a567
Replace if with switch in OrderedMap.Storable()
fxamacker Sep 21, 2023
5194eee
Add comment about encoding size
fxamacker Sep 21, 2023
7375b82
Add comments for parentUpdater
fxamacker Sep 21, 2023
d49d6b4
Add more comments for Array and OrderedMap
fxamacker Sep 21, 2023
f56232f
Add comment about ValueID and SlabID
fxamacker Sep 21, 2023
24aa117
Validate map key/value size <= max limit in tests
fxamacker Sep 22, 2023
15bde12
Validate ValueID and SlabID for arrays in tests
fxamacker Sep 22, 2023
27aa4cb
Validate ValueID and SlabID for maps in tests
fxamacker Sep 22, 2023
5d8ff98
Validate address of inlined array/map in tests
fxamacker Sep 22, 2023
6cd8c0c
Verify inlinability of not inlined values in tests
fxamacker Sep 22, 2023
43edeca
Refactor array validation
fxamacker Sep 24, 2023
119f6c5
Test inlined array slabs are not in storage
fxamacker Sep 24, 2023
efcb2a2
Refactor map validation
fxamacker Sep 24, 2023
5af0bc4
Test inlined map slabs are not in storage
fxamacker Sep 24, 2023
a71e7f8
Refactor array and map validation
fxamacker Sep 24, 2023
2d8a4e8
Refactor array and map serialization validation
fxamacker Sep 24, 2023
ed36d65
Refactor array and map validation
fxamacker Sep 25, 2023
04f93a2
Improve tests to compare child array elements
fxamacker Sep 26, 2023
00a6df8
Improve tests to compare child map elements
fxamacker Sep 26, 2023
d3de291
Add ReadOnly iterators and refactor other iterators
fxamacker Jul 18, 2023
e88a73e
Support value mutation from non-readonly iterators
fxamacker Sep 29, 2023
a75e388
Add more comments about how atree inlining works
fxamacker Oct 1, 2023
042eb68
Uninline slab when it is overwritten or removed
fxamacker Oct 1, 2023
f17354c
Refactor inlining functions
fxamacker Oct 2, 2023
1a2e69f
Prune Array.mutableElementIndex in Set and Remove
fxamacker Oct 2, 2023
63ea7a7
No-op on parentUpdater() if child isn't in parent
fxamacker Oct 2, 2023
53a716f
Replace magic numbers with constants
fxamacker Oct 2, 2023
7a2aeb1
Use unsafe.Sizeof instead of magic number
fxamacker Oct 2, 2023
e6fa347
Add comment for potential overlapping tag nums in Cadence
fxamacker Oct 2, 2023
03acd49
Create Array.mutableElementIndex lazily
fxamacker Oct 2, 2023
90be7ac
Refactor to use SlabIndex instead of [8]byte
fxamacker Oct 2, 2023
f3fdb2d
Replace magic number with constant
fxamacker Oct 2, 2023
2775ff5
Refactor to use same variable in type switches
fxamacker Oct 2, 2023
836eb70
Reuse buffer from pool when encoding elements
fxamacker Oct 2, 2023
4b3c26c
Rename composite to compactMap
fxamacker Oct 2, 2023
3f87ec5
Check duplicate SlabID in inlined slabs in tests
fxamacker Oct 3, 2023
903fc13
Create Go maps in inlinedExtraData lazily
fxamacker Oct 3, 2023
8d02bbc
Rename variables for clarity
fxamacker Oct 3, 2023
5df8b16
Merge branch 'fxamacker/inline-array-and-map' into fxamacker/add-keyc…
fxamacker Oct 3, 2023
2a6091a
Add more comments
fxamacker Oct 3, 2023
c07907d
Add MapIterator.CanMutate() predicate function
fxamacker Oct 3, 2023
f671189
Add ArrayIterator.CanMutate() predicate function
fxamacker Oct 3, 2023
5f1de0a
Merge pull request #345 from onflow/fxamacker/add-keycomparator-and-h…
fxamacker Oct 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,098 changes: 978 additions & 120 deletions array.go
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions array_bench_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,7 @@ func benchmarkNewArrayFromAppend(b *testing.B, initialArraySize int) {
for i := 0; i < b.N; i++ {
copied, _ := NewArray(storage, array.Address(), array.Type())

_ = array.Iterate(func(value Value) (bool, error) {
_ = array.IterateReadOnly(func(value Value) (bool, error) {
_ = copied.Append(value)
return true, nil
})
Expand All @@ -379,7 +379,7 @@ func benchmarkNewArrayFromBatchData(b *testing.B, initialArraySize int) {
b.StartTimer()

for i := 0; i < b.N; i++ {
iter, err := array.Iterator()
iter, err := array.ReadOnlyIterator()
require.NoError(b, err)

copied, _ := NewArrayFromBatchData(storage, array.Address(), array.Type(), func() (Value, error) {
Expand Down
Loading
Loading