Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Binary heap structure #5084

Merged
merged 38 commits into from
Jul 23, 2024
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
4fec530
implement binary heap
Amxx Jun 16, 2024
8fa2eeb
codespell & lib naming
Amxx Jun 16, 2024
792fcba
tests
Amxx Jun 16, 2024
0c86005
fix fuzzing tests
Amxx Jun 16, 2024
248baf6
codespell
Amxx Jun 16, 2024
53db2ab
update
Amxx Jun 17, 2024
945e0f4
procedural generation
Amxx Jun 17, 2024
df82b15
testing
Amxx Jun 17, 2024
8b965fc
overflow handling
Amxx Jun 21, 2024
e952cf6
add replace and changeset
Amxx Jun 21, 2024
f5fa274
rename top -> peek
Amxx Jun 21, 2024
1f0fef0
internal renaming
Amxx Jun 21, 2024
d0972a3
codespell
Amxx Jun 21, 2024
8e3dda6
regenerate
Amxx Jun 21, 2024
38e1813
auto regenerate
Amxx Jun 21, 2024
02f224d
Update .githooks/pre-push
Amxx Jun 21, 2024
7e88481
up
Amxx Jun 21, 2024
a46cc63
Merge branch 'master' into struct/heap
Amxx Jun 21, 2024
b2fda31
up
Amxx Jun 21, 2024
516f1ca
tests
Amxx Jun 21, 2024
cf1278e
Update test/utils/structs/Heap.test.js
Amxx Jun 21, 2024
5f15d1c
Update test/utils/structs/Heap.test.js
Amxx Jun 21, 2024
32e9b49
Apply suggestions from code review
Amxx Jun 27, 2024
c083d79
regenrate
Amxx Jun 27, 2024
0e6ada0
Merge branch 'master' into struct/heap
Amxx Jul 3, 2024
7c98102
update inline comments
Amxx Jul 15, 2024
a1767d4
update
Amxx Jul 15, 2024
1c1e84b
Address comment for the PR
Amxx Jul 16, 2024
0e7fe7a
rewrite Arrays.sol to use uint256[] as the default, and use Comparato…
Amxx Jul 17, 2024
d495859
Update scripts/generate/templates/Heap.js
Amxx Jul 18, 2024
fe8e902
regenerate
Amxx Jul 18, 2024
3abeb84
Add docs
ernestognw Jul 23, 2024
8801d98
Update scripts/generate/templates/Heap.js
Amxx Jul 23, 2024
f78df0c
Apply suggestions from code review
Amxx Jul 23, 2024
bb37dfb
fix generation + change key type
Amxx Jul 23, 2024
1fb4b81
more invariant check
Amxx Jul 23, 2024
d3308c4
Update scripts/generate/templates/Heap.js
ernestognw Jul 23, 2024
5b07512
Generate
ernestognw Jul 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fluffy-buses-jump.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'openzeppelin-solidity': minor
---

`Comparator`: A library of comparator functions, useful for customizing the behavior of the Heap structure.
5 changes: 5 additions & 0 deletions .changeset/great-pianos-work.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'openzeppelin-solidity': minor
---

`Heap`: A data structure that implements a heap-based priority queue.
1 change: 1 addition & 0 deletions .githooks/pre-push
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,6 @@
set -euo pipefail

if [ "${CI:-"false"}" != "true" ]; then
npm run test:generation
npm run lint
fi
1 change: 1 addition & 0 deletions contracts/mocks/Stateless.sol
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import {ERC165} from "../utils/introspection/ERC165.sol";
import {ERC165Checker} from "../utils/introspection/ERC165Checker.sol";
import {ERC1967Utils} from "../proxy/ERC1967/ERC1967Utils.sol";
import {ERC721Holder} from "../token/ERC721/utils/ERC721Holder.sol";
import {Heap} from "../utils/structs/Heap.sol";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not adding Comparators here?

Suggested change
import {Heap} from "../utils/structs/Heap.sol";
import {Heap} from "../utils/structs/Heap.sol";
import {Comparators} from "../utils/Comparators.sol";

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was overlooked, and since there are currently no tests for the Comparators library, it missing did not triger any issue.

Lets start by discussing weither we want the comparator library or not ... then we can add tests and solve that

import {Math} from "../utils/math/Math.sol";
import {MerkleProof} from "../utils/cryptography/MerkleProof.sol";
import {MessageHashUtils} from "../utils/cryptography/MessageHashUtils.sol";
Expand Down
72 changes: 34 additions & 38 deletions contracts/utils/Arrays.sol
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

pragma solidity ^0.8.20;

import {Comparators} from "./Comparators.sol";
import {SlotDerivation} from "./SlotDerivation.sol";
import {StorageSlot} from "./StorageSlot.sol";
import {Math} from "./math/Math.sol";
Expand All @@ -16,7 +17,7 @@ library Arrays {
using StorageSlot for bytes32;

/**
* @dev Sort an array of bytes32 (in memory) following the provided comparator function.
* @dev Sort an array of uint256 (in memory) following the provided comparator function.
*
* This function does the sorting "in place", meaning that it overrides the input. The object is returned for
* convenience, but that returned value can be discarded safely if the caller has a memory pointer to the array.
Expand All @@ -27,18 +28,18 @@ library Arrays {
* consume more gas than is available in a block, leading to potential DoS.
*/
function sort(
bytes32[] memory array,
function(bytes32, bytes32) pure returns (bool) comp
) internal pure returns (bytes32[] memory) {
uint256[] memory array,
function(uint256, uint256) pure returns (bool) comp
) internal pure returns (uint256[] memory) {
_quickSort(_begin(array), _end(array), comp);
return array;
}

/**
* @dev Variant of {sort} that sorts an array of bytes32 in increasing order.
* @dev Variant of {sort} that sorts an array of uint256 in increasing order.
*/
function sort(bytes32[] memory array) internal pure returns (bytes32[] memory) {
sort(array, _defaultComp);
function sort(uint256[] memory array) internal pure returns (uint256[] memory) {
sort(array, Comparators.lt);
return array;
}

Expand All @@ -57,20 +58,20 @@ library Arrays {
address[] memory array,
function(address, address) pure returns (bool) comp
) internal pure returns (address[] memory) {
sort(_castToBytes32Array(array), _castToBytes32Comp(comp));
sort(_castToUint256Array(array), _castToUint256Comp(comp));
return array;
}

/**
* @dev Variant of {sort} that sorts an array of address in increasing order.
*/
function sort(address[] memory array) internal pure returns (address[] memory) {
sort(_castToBytes32Array(array), _defaultComp);
sort(_castToUint256Array(array), Comparators.lt);
return array;
}

/**
* @dev Sort an array of uint256 (in memory) following the provided comparator function.
* @dev Sort an array of bytes32 (in memory) following the provided comparator function.
*
* This function does the sorting "in place", meaning that it overrides the input. The object is returned for
* convenience, but that returned value can be discarded safely if the caller has a memory pointer to the array.
Expand All @@ -81,18 +82,18 @@ library Arrays {
* consume more gas than is available in a block, leading to potential DoS.
*/
function sort(
uint256[] memory array,
function(uint256, uint256) pure returns (bool) comp
) internal pure returns (uint256[] memory) {
sort(_castToBytes32Array(array), _castToBytes32Comp(comp));
bytes32[] memory array,
function(bytes32, bytes32) pure returns (bool) comp
) internal pure returns (bytes32[] memory) {
sort(_castToUint256Array(array), _castToUint256Comp(comp));
return array;
}

/**
* @dev Variant of {sort} that sorts an array of uint256 in increasing order.
* @dev Variant of {sort} that sorts an array of bytes32 in increasing order.
*/
function sort(uint256[] memory array) internal pure returns (uint256[] memory) {
sort(_castToBytes32Array(array), _defaultComp);
function sort(bytes32[] memory array) internal pure returns (bytes32[] memory) {
sort(_castToUint256Array(array), Comparators.lt);
return array;
}

Expand All @@ -105,12 +106,12 @@ library Arrays {
* IMPORTANT: Memory locations between `begin` and `end` are not validated/zeroed. This function should
* be used only if the limits are within a memory array.
*/
function _quickSort(uint256 begin, uint256 end, function(bytes32, bytes32) pure returns (bool) comp) private pure {
function _quickSort(uint256 begin, uint256 end, function(uint256, uint256) pure returns (bool) comp) private pure {
unchecked {
if (end - begin < 0x40) return;

// Use first element as pivot
bytes32 pivot = _mload(begin);
uint256 pivot = _mload(begin);
// Position where the pivot should be at the end of the loop
uint256 pos = begin;

Expand All @@ -132,7 +133,7 @@ library Arrays {
/**
* @dev Pointer to the memory location of the first element of `array`.
*/
function _begin(bytes32[] memory array) private pure returns (uint256 ptr) {
function _begin(uint256[] memory array) private pure returns (uint256 ptr) {
/// @solidity memory-safe-assembly
assembly {
ptr := add(array, 0x20)
Expand All @@ -143,16 +144,16 @@ library Arrays {
* @dev Pointer to the memory location of the first memory word (32bytes) after `array`. This is the memory word
* that comes just after the last element of the array.
*/
function _end(bytes32[] memory array) private pure returns (uint256 ptr) {
function _end(uint256[] memory array) private pure returns (uint256 ptr) {
unchecked {
return _begin(array) + array.length * 0x20;
}
}

/**
* @dev Load memory word (as a bytes32) at location `ptr`.
* @dev Load memory word (as a uint256) at location `ptr`.
*/
function _mload(uint256 ptr) private pure returns (bytes32 value) {
function _mload(uint256 ptr) private pure returns (uint256 value) {
assembly {
value := mload(ptr)
}
Expand All @@ -170,38 +171,33 @@ library Arrays {
}
}

/// @dev Comparator for sorting arrays in increasing order.
function _defaultComp(bytes32 a, bytes32 b) private pure returns (bool) {
return a < b;
}

/// @dev Helper: low level cast address memory array to uint256 memory array
function _castToBytes32Array(address[] memory input) private pure returns (bytes32[] memory output) {
function _castToUint256Array(address[] memory input) private pure returns (uint256[] memory output) {
assembly {
output := input
}
}

/// @dev Helper: low level cast uint256 memory array to uint256 memory array
function _castToBytes32Array(uint256[] memory input) private pure returns (bytes32[] memory output) {
/// @dev Helper: low level cast bytes32 memory array to uint256 memory array
function _castToUint256Array(bytes32[] memory input) private pure returns (uint256[] memory output) {
assembly {
output := input
}
}

/// @dev Helper: low level cast address comp function to bytes32 comp function
function _castToBytes32Comp(
/// @dev Helper: low level cast address comp function to uint256 comp function
function _castToUint256Comp(
function(address, address) pure returns (bool) input
) private pure returns (function(bytes32, bytes32) pure returns (bool) output) {
) private pure returns (function(uint256, uint256) pure returns (bool) output) {
assembly {
output := input
}
}

/// @dev Helper: low level cast uint256 comp function to bytes32 comp function
function _castToBytes32Comp(
function(uint256, uint256) pure returns (bool) input
) private pure returns (function(bytes32, bytes32) pure returns (bool) output) {
/// @dev Helper: low level cast bytes32 comp function to uint256 comp function
function _castToUint256Comp(
function(bytes32, bytes32) pure returns (bool) input
) private pure returns (function(uint256, uint256) pure returns (bool) output) {
assembly {
output := input
}
Expand Down
13 changes: 13 additions & 0 deletions contracts/utils/Comparators.sol
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of having comparators so that the heap is customizable. However, this library feels odd. Is this something we see value in providing on its own file? I'd rather keep it undocumented and inside Heap.sol

Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
// SPDX-License-Identifier: MIT

pragma solidity ^0.8.20;

library Comparators {
function lt(uint256 a, uint256 b) internal pure returns (bool) {
return a < b;
}

function gt(uint256 a, uint256 b) internal pure returns (bool) {
return a > b;
}
}
6 changes: 6 additions & 0 deletions contracts/utils/README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Miscellaneous contracts and libraries containing utility functions you can use t
* {DoubleEndedQueue}: An implementation of a https://en.wikipedia.org/wiki/Double-ended_queue[double ended queue] whose values can be removed added or remove from both sides. Useful for FIFO and LIFO structures.
* {CircularBuffer}: A data structure to store the last N values pushed to it.
* {Checkpoints}: A data structure to store values mapped to an strictly increasing key. Can be used for storing and accessing values over time.
* {Heap}: A library that implements a https://en.wikipedia.org/wiki/Binary_heap[binary heap] in storage.
* {MerkleTree}: A library with https://wikipedia.org/wiki/Merkle_Tree[Merkle Tree] data structures and helper functions.
* {Create2}: Wrapper around the https://blog.openzeppelin.com/getting-the-most-out-of-create2/[`CREATE2` EVM opcode] for safe use without having to deal with low-level assembly.
* {Address}: Collection of functions for overloading Solidity's https://docs.soliditylang.org/en/latest/types.html#address[`address`] type.
Expand All @@ -38,6 +39,7 @@ Miscellaneous contracts and libraries containing utility functions you can use t
* {Context}: An utility for abstracting the sender and calldata in the current execution context.
* {Packing}: A library for packing and unpacking multiple values into bytes32
* {Panic}: A library to revert with https://docs.soliditylang.org/en/v0.8.20/control-structures.html#panic-via-assert-and-error-via-require[Solidity panic codes].
* {Comparators}: A library that contains comparator functions to use with with the {Heap} library.

[NOTE]
====
Expand Down Expand Up @@ -102,6 +104,8 @@ Ethereum contracts have no native concept of an interface, so applications must

{{Checkpoints}}

{{Heap}}

{{MerkleTree}}

== Libraries
Expand Down Expand Up @@ -129,3 +133,5 @@ Ethereum contracts have no native concept of an interface, so applications must
{{Packing}}

{{Panic}}

{{Comparators}}
Loading