-
Notifications
You must be signed in to change notification settings - Fork 11.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a double ended queue #3153
Merged
Merged
Add a double ended queue #3153
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
d3854ee
add vector, lifo and fifo structures
Amxx b5358a9
fix lint
Amxx 6140b20
need more memory for coverage
Amxx 1157045
remove Vector wrappers and gas optimization
Amxx 9e9dd6d
refactor Vector testing
Amxx bfb6d57
revert package.json changes
Amxx c7ec6fd
rename to DoubleEndedQueue
frangio 933337e
rename and refactor
frangio b7fb096
refactor tests and expand coverage
frangio 17a8fea
test for custom errors
frangio 8eea114
Merge branch 'master' into feature/vector
frangio 595d5ef
add changelog entry
frangio 914e1df
add docs
frangio c5ef056
add sample code and note about storage vs. memory
frangio 1cf95f1
add available since
frangio adc9a5a
lint
frangio 0649a31
use underscore for struct members
frangio bbb091d
add struct documentation
frangio 3aff9f0
remove SafeCast in length
frangio 35064a7
Merge branch 'master' into feature/vector
frangio 248d479
rename i -> index and improve docs
frangio File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
// SPDX-License-Identifier: MIT | ||
|
||
pragma solidity ^0.8.0; | ||
|
||
import "../utils/structs/DoubleEndedQueue.sol"; | ||
|
||
// Bytes32Deque | ||
contract Bytes32DequeMock { | ||
using DoubleEndedQueue for DoubleEndedQueue.Bytes32Deque; | ||
|
||
event OperationResult(bytes32 value); | ||
|
||
DoubleEndedQueue.Bytes32Deque private _vector; | ||
|
||
function pushBack(bytes32 value) public { | ||
_vector.pushBack(value); | ||
} | ||
|
||
function pushFront(bytes32 value) public { | ||
_vector.pushFront(value); | ||
} | ||
|
||
function popFront() public returns (bytes32) { | ||
bytes32 value = _vector.popFront(); | ||
emit OperationResult(value); | ||
return value; | ||
} | ||
|
||
function popBack() public returns (bytes32) { | ||
bytes32 value = _vector.popBack(); | ||
emit OperationResult(value); | ||
return value; | ||
} | ||
|
||
function front() public view returns (bytes32) { | ||
return _vector.front(); | ||
} | ||
|
||
function back() public view returns (bytes32) { | ||
return _vector.back(); | ||
} | ||
|
||
function at(uint256 i) public view returns (bytes32) { | ||
return _vector.at(i); | ||
} | ||
|
||
function clear() public { | ||
_vector.clear(); | ||
} | ||
|
||
function length() public view returns (uint256) { | ||
return _vector.length(); | ||
} | ||
|
||
function empty() public view returns (bool) { | ||
return _vector.empty(); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,165 @@ | ||
// SPDX-License-Identifier: MIT | ||
pragma solidity ^0.8.4; | ||
|
||
import "../math/SafeCast.sol"; | ||
|
||
/** | ||
* @dev A sequence of items with the ability to efficiently push and pop items (i.e. insert and remove) on both ends of | ||
* the sequence (called front and back). Among other access patterns, it can be used to implement efficient LIFO and | ||
* FIFO queues. Storage use is optimized, and all operations are O(1) constant time. This includes {clear}, given that | ||
* the existing queue contents are left in storage. | ||
* | ||
* The struct is called `Bytes32Deque`. Other types can be cast to and from `bytes32`. This data structure can only be | ||
* used in storage, and not in memory. | ||
* ``` | ||
* DoubleEndedQueue.Bytes32Deque queue; | ||
* ``` | ||
* | ||
* _Available since v4.6._ | ||
*/ | ||
library DoubleEndedQueue { | ||
/** | ||
* @dev An operation (e.g. {front}) couldn't be completed due to the queue being empty. | ||
*/ | ||
error Empty(); | ||
|
||
/** | ||
* @dev An operation (e.g. {at}) could't be completed due to an index being out of bounds. | ||
*/ | ||
error OutOfBounds(); | ||
|
||
/** | ||
* @dev Indices are signed integers because the queue can grow in any direction. They are 128 bits so begin and end | ||
* are packed in a single storage slot for efficient access. Since the items are added one at a time we can safely | ||
* assume that these 128-bit indices will not overflow, and use unchecked arithmetic. | ||
* | ||
* Struct members have an underscore prefix indicating that they are "private" and should not be read or written to | ||
* directly. Use the functions provided below instead. Modifying the struct manually may violate assumptions and | ||
* lead to unexpected behavior. | ||
* | ||
* Indices are in the range [begin, end) which means the first item is at data[begin] and the last item is at | ||
* data[end - 1]. | ||
*/ | ||
struct Bytes32Deque { | ||
int128 _begin; | ||
int128 _end; | ||
mapping(int128 => bytes32) _data; | ||
} | ||
|
||
/** | ||
* @dev Inserts an item at the end of the queue. | ||
*/ | ||
function pushBack(Bytes32Deque storage deque, bytes32 value) internal { | ||
int128 backIndex = deque._end; | ||
deque._data[backIndex] = value; | ||
unchecked { | ||
deque._end = backIndex + 1; | ||
} | ||
} | ||
|
||
/** | ||
* @dev Removes the item at the end of the queue and returns it. | ||
* | ||
* Reverts with `Empty` if the queue is empty. | ||
*/ | ||
function popBack(Bytes32Deque storage deque) internal returns (bytes32 value) { | ||
if (empty(deque)) revert Empty(); | ||
int128 backIndex; | ||
unchecked { | ||
backIndex = deque._end - 1; | ||
} | ||
value = deque._data[backIndex]; | ||
delete deque._data[backIndex]; | ||
deque._end = backIndex; | ||
} | ||
|
||
/** | ||
* @dev Inserts an item at the beginning of the queue. | ||
*/ | ||
function pushFront(Bytes32Deque storage deque, bytes32 value) internal { | ||
int128 frontIndex; | ||
unchecked { | ||
frontIndex = deque._begin - 1; | ||
} | ||
deque._data[frontIndex] = value; | ||
deque._begin = frontIndex; | ||
} | ||
|
||
/** | ||
* @dev Removes the item at the beginning of the queue and returns it. | ||
* | ||
* Reverts with `Empty` if the queue is empty. | ||
*/ | ||
function popFront(Bytes32Deque storage deque) internal returns (bytes32 value) { | ||
if (empty(deque)) revert Empty(); | ||
int128 frontIndex = deque._begin; | ||
value = deque._data[frontIndex]; | ||
delete deque._data[frontIndex]; | ||
unchecked { | ||
deque._begin = frontIndex + 1; | ||
} | ||
} | ||
|
||
/** | ||
* @dev Returns the item at the beginning of the queue. | ||
*/ | ||
function front(Bytes32Deque storage deque) internal view returns (bytes32 value) { | ||
if (empty(deque)) revert Empty(); | ||
int128 frontIndex = deque._begin; | ||
return deque._data[frontIndex]; | ||
} | ||
|
||
/** | ||
* @dev Returns the item at the end of the queue. | ||
*/ | ||
function back(Bytes32Deque storage deque) internal view returns (bytes32 value) { | ||
if (empty(deque)) revert Empty(); | ||
int128 backIndex; | ||
unchecked { | ||
backIndex = deque._end - 1; | ||
} | ||
return deque._data[backIndex]; | ||
} | ||
|
||
/** | ||
* @dev Return the item at a position in the queue given by `index`, with the first item at 0 and last item at | ||
* `length(deque) - 1`. | ||
* | ||
* Reverts with `OutOfBounds` if the index is out of bounds. | ||
*/ | ||
function at(Bytes32Deque storage deque, uint256 index) internal view returns (bytes32 value) { | ||
// int256(deque._begin) is a safe upcast | ||
int128 idx = SafeCast.toInt128(int256(deque._begin) + SafeCast.toInt256(index)); | ||
if (idx >= deque._end) revert OutOfBounds(); | ||
return deque._data[idx]; | ||
} | ||
|
||
/** | ||
* @dev Resets the queue back to being empty. | ||
* | ||
* NOTE: The current items are left behind in storage. This does not affect the functioning of the queue, but misses | ||
* out on potential gas refunds. | ||
*/ | ||
function clear(Bytes32Deque storage deque) internal { | ||
deque._begin = 0; | ||
deque._end = 0; | ||
} | ||
|
||
/** | ||
* @dev Returns the number of items in the queue. | ||
*/ | ||
function length(Bytes32Deque storage deque) internal view returns (uint256) { | ||
// The interface preserves the invariant that begin <= end so we assume this will not overflow. | ||
// We also assume there are at most int256.max items in the queue. | ||
unchecked { | ||
return uint256(int256(deque._end) - int256(deque._begin)); | ||
} | ||
} | ||
|
||
/** | ||
* @dev Returns true if the queue is empty. | ||
*/ | ||
function empty(Bytes32Deque storage deque) internal view returns (bool) { | ||
return deque._end <= deque._begin; | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,108 @@ | ||
const { expectEvent } = require('@openzeppelin/test-helpers'); | ||
const { expect } = require('chai'); | ||
|
||
const Bytes32DequeMock = artifacts.require('Bytes32DequeMock'); | ||
|
||
/** Rebuild the content of the deque as a JS array. */ | ||
async function getContent (deque) { | ||
const length = await deque.length().then(bn => bn.toNumber()); | ||
const values = await Promise.all(Array(length).fill().map((_, i) => deque.at(i))); | ||
return values; | ||
} | ||
|
||
/** Revert handler that supports custom errors. */ | ||
async function expectRevert (promise, reason) { | ||
try { | ||
await promise; | ||
expect.fail('Expected promise to throw but it didn\'t'); | ||
} catch (error) { | ||
if (reason) { | ||
expect(error.message).to.include(reason); | ||
} | ||
} | ||
} | ||
|
||
contract('DoubleEndedQueue', function (accounts) { | ||
const bytesA = '0xdeadbeef'.padEnd(66, '0'); | ||
const bytesB = '0x0123456789'.padEnd(66, '0'); | ||
const bytesC = '0x42424242'.padEnd(66, '0'); | ||
const bytesD = '0x171717'.padEnd(66, '0'); | ||
|
||
beforeEach(async function () { | ||
this.deque = await Bytes32DequeMock.new(); | ||
}); | ||
|
||
describe('when empty', function () { | ||
it('getters', async function () { | ||
expect(await this.deque.empty()).to.be.equal(true); | ||
expect(await getContent(this.deque)).to.have.ordered.members([]); | ||
}); | ||
|
||
it('reverts on accesses', async function () { | ||
await expectRevert(this.deque.popBack(), 'Empty()'); | ||
await expectRevert(this.deque.popFront(), 'Empty()'); | ||
await expectRevert(this.deque.back(), 'Empty()'); | ||
await expectRevert(this.deque.front(), 'Empty()'); | ||
}); | ||
}); | ||
|
||
describe('when not empty', function () { | ||
beforeEach(async function () { | ||
await this.deque.pushBack(bytesB); | ||
await this.deque.pushFront(bytesA); | ||
await this.deque.pushBack(bytesC); | ||
this.content = [ bytesA, bytesB, bytesC ]; | ||
}); | ||
|
||
it('getters', async function () { | ||
expect(await this.deque.empty()).to.be.equal(false); | ||
expect(await this.deque.length()).to.be.bignumber.equal(this.content.length.toString()); | ||
expect(await this.deque.front()).to.be.equal(this.content[0]); | ||
expect(await this.deque.back()).to.be.equal(this.content[this.content.length - 1]); | ||
expect(await getContent(this.deque)).to.have.ordered.members(this.content); | ||
}); | ||
|
||
it('out of bounds access', async function () { | ||
await expectRevert(this.deque.at(this.content.length), 'OutOfBounds()'); | ||
}); | ||
|
||
describe('push', function () { | ||
it('front', async function () { | ||
await this.deque.pushFront(bytesD); | ||
this.content.unshift(bytesD); // add element at the begining | ||
|
||
expect(await getContent(this.deque)).to.have.ordered.members(this.content); | ||
}); | ||
|
||
it('back', async function () { | ||
await this.deque.pushBack(bytesD); | ||
this.content.push(bytesD); // add element at the end | ||
|
||
expect(await getContent(this.deque)).to.have.ordered.members(this.content); | ||
}); | ||
}); | ||
|
||
describe('pop', function () { | ||
it('front', async function () { | ||
const value = this.content.shift(); // remove first element | ||
expectEvent(await this.deque.popFront(), 'OperationResult', { value }); | ||
|
||
expect(await getContent(this.deque)).to.have.ordered.members(this.content); | ||
}); | ||
|
||
it('back', async function () { | ||
const value = this.content.pop(); // remove last element | ||
expectEvent(await this.deque.popBack(), 'OperationResult', { value }); | ||
|
||
expect(await getContent(this.deque)).to.have.ordered.members(this.content); | ||
}); | ||
}); | ||
|
||
it('clear', async function () { | ||
await this.deque.clear(); | ||
|
||
expect(await this.deque.empty()).to.be.equal(true); | ||
expect(await getContent(this.deque)).to.have.ordered.members([]); | ||
}); | ||
}); | ||
}); |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO, the previous implementation
was way more easy to read/understand (same argument apply to all push and pop functions)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pre/post inc/decrements are quick to read, but definitely not easy...
The statement
deque.data[deque.end++] = value
is too dense: there are two different storage updates, and it's not immediately clear what value is used to index the data mapping.I tried a few alternatives and ended up going with the current option. I think this code is quite ok:
The
unchecked
makes it harder to read. Perhaps if we put everything in theunchecked
block it would look better?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I definitely think we want the
+1
unchecked.Putting everything in the unchecked block might make it cleaner to read (all the logic is indented at the same level)