-
Notifications
You must be signed in to change notification settings - Fork 174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
voting: Overhaul the voting system #1809
voting: Overhaul the voting system #1809
Conversation
// Order the output address groups by total amount. For wallets with | ||
// more outputs than the number of outputs that will fit in a voting | ||
// transaction, we want to choose the largest addresses first, so we | ||
// sort in descending order. | ||
// | ||
// TODO: Ordering by address is not perfect because an address could | ||
// contain one large output and hundreds of small outputs. We should | ||
// use a more accurate order based on something like the average for | ||
// the output amounts associated with the address. | ||
// | ||
std::sort(ordered.begin(), ordered.end(), std::greater<AddressOutputs>()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think a better way to do this would be to pivot the outputs and the addresses and then order by outputs descending without specifically in regards to addresses. The addresses referred to by the outputs chosen is implicitly the set of addresses that would be included. Note that not all UTXO's would be included for a given address, but that is ok... that is exactly what we want. I haven't thought about the ramifications of my suggestion on other parts of your code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think using the average as an alternative is going to be particularly helpful, because many wallets are going to have a UTXO size that is exponentially distributed, with a size cutoff roughly at 4-5k (which is what the stake splitter for stake optimization generates with the current difficulty).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If changing this doesn't break validation code, then optimizing this is a leisure...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is just on the creation side, so changing the output selection won't impact consensus. You're right about using an average... we really need to trim the outputs before grouping them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's see how the existing design works in testing...
Not finished with the review yet. Very good so far... |
0c98527
to
6059b53
Compare
This reverts commit 170fa57.
This removes the magnitude-only, CPID count, and participants count voting weight types from the options available in the poll creation dialog.
This overload enables client code to configure aspects of the transaction that contains contract message. The other version of the function accepts a contract object argument only. This makes it difficult to design contracts that depend on aspects of the transactions that contain them.
This method returned a floating-point number. This switches the return value to an integer. The upcoming changes to the voting system rely on the total magnitude to calculate voting weight. The use of the integer return type avoids floating-point discrepancies and allows the network to potentially integrate the voting system into the protocol in future enhancements. Note that this change truncates the fractional portion of the network- wide magnitude from the return value. It is not significant to current voting weight calculations.
This adds a set of classes that replace the legacy voting functions. These classes manage all aspects of the voting system.
Also removes these functions: - GetEarliestStakeTime() - GetEarliestWalletTransaction()
This allows a user to unlock the wallet to send a poll or vote contract from the GUI if they forget to unlock it beforehand.
This enables the GUI to label poll transactions in the transaction list. Before, the transaction appeared as a blank "send" transaction which may confuse users.
Added - Backport newer uint256 types from Bitcoin #1570 (@cyrossignol) - Implement project level rain for rainbymagnitude #1580 (@jamescowens) - Upgrade utilities (Update checker and snapshot downloader/application) #1576 (@iFoggz) - Provide fees collected in the block by the miner #1601 (@iFoggz) - Add support for generating legacy superblocks from scraper stats #1603 (@cyrossignol) - Port of the Bitcoin Logger to Gridcoin #1600 (@jamescowens) - Implement zapwallettxes #1605 (@jamescowens) - Implements a global event filter to suppress help question mark #1609 (@jamescowens) - Add next target difficulty to RPC output #1615 (@cyrossignol) - Add caching for block hashes to CBlock #1624 (@cyrossignol) - Make toolbars and tray icon red for testnet #1637 (@jamescowens) - Add an rpc call convergencereport #1643 (@jamescowens) - Implement newline filter on config file read in #1645 (@jamescowens) - Implement beacon status icon/button #1646 (@jamescowens) - Add gridcointestnet.png #1649 (@caraka) - Add precision to support magnitudes less than 1 #1651 (@cyrossignol) - Replace research accrual calculations with superblock snapshots #1657 (@cyrossignol) - Publish example gridcoinresearch.conf as a md document to the doc directory #1662 (@jamescowens) - Add options checkbox to disable transaction notifications #1666 (@jamescowens) - Add support for self-service beacon deletion #1695 (@cyrossignol) - Add support for type-specific contract fee amounts #1698 (@cyrossignol) - Add verifiedbeaconreport and pendingbeaconreport #1696 (@jamescowens) - Add preliminary testing option for block v11 height on testnet #1706 (@cyrossignol) - Add verified beacons manifest part to superblock validator #1711 (@cyrossignol) - Implement beacon, vote, and superblock display categories/icons in UI transaction model #1717 (@jamescowens) - neuralnet: Add integrity checking to researcher accrual snapshot registry #1727 (@jamescowens) - Add workaround for scrypt assembly on macOS #1740 (@cyrossignol) - gui: Build onboarding/beacon wizard #1739 (@cyrossignol) - doc: Add CONTRIBUTING.md from bitcoin #1723 (@div72) - rpc: Implement inspectaccrualsnapshot and parseaccrualsnapshotfile #1744 (@jamescowens) - scraper: Add disk based state backing for verified beacon list in scraper #1751 (@jamescowens) - Add ability to recover beacon in block version 11+ #1768 (@cyrossignol) - refactor: Add transaction context to contract handlers #1777 (@cyrossignol) - gui: Add context for when BOINC is attached to a pool #1775 (@cyrossignol) - doc: Clarify what to do if PR in multiple categories (for CONTRIBUTING.md) #1798 (@RoboticMind) - qt: Add option to choose not to start the wallet minimized #1804 (@jamescowens) - superblock: Add check for OutOfSyncByAge to SuperblockValidator::Validate #1806 (@jamescowens) - contract: Standardize contract validation and add block context #1808 (@cyrossignol) - add seed.gridcoin.pl to default config #1812 (@wilkart) - gui: Implement sidestake send display #1813 (@jamescowens) - gui: Add pool/investor pages to researcher wizard #1819 (@cyrossignol) - ci: Port lint scripts from Bitcoin #1823 (@div72) - doc: Create basic readme in contrib #1826 (@RoboticMind) - gui: Implement TransactionRecord::Message #1829 (@jamescowens) - rpc: Add private_key_available to beaconstatus #1833 (@a123b) - gui: Validate email address in researcher wizard #1840 (@a123b) - rpc: Add "getrawwallettransaction" RPC function #1842 (@cyrossignol) - consensus: Set block version 11 threshold height for mainnet #1862 (@cyrossignol) Changed - Upgrade LevelDB from v1.17 to v1.20 #1562 (@cyrossignol) - Re-enable scrypt optimizations #1450 (@denravonska) - Derive CScript from prevector type (optimization) #1554 (@cyrossignol) - Disable quorum for grandfathered blocks to speed up sync #1568 (@cyrossignol) - Refactor hashBoinc for binary claim contexts #1558 (@cyrossignol) - integrated_scraper_2 branch tracking PR #1559 (@jamescowens) - Upgrade depends - OpenSSL to 1.1.1d #1581 (@jamescowens) - Ubuntu 19.10 fixes #1590 (@denravonska) - Force a re-parse of legacy claims in generated blocks #1592 (@cyrossignol) - Improve the "versionreport" RPC output #1595 (@cyrossignol) - Overhaul the core tally and accrual system #1583 (@cyrossignol) - Overhaul the superblock quorum system #1597 (@cyrossignol) - Add more data to the "superblocks" RPC output #1599 (@cyrossignol) - Update Windows Build doc #1606 (@barton2526) - Change the order of calls in gridcoinresearchd.cpp to optimize rpc shunt path #1610 (@jamescowens) - Change staking tooltip to display frequency #1611 (@jamescowens) - Enhancements to ETTS #1442 (@jamescowens) - Standardize money values as integers #1614 (@cyrossignol) - Clean up and optimize legacy coin age code #1616 (@cyrossignol) - Some scraper cleanups #1620 (@jamescowens) - Reorganize accrual code and fix 6-month cutoff #1630 (@cyrossignol) - Update Copyright years #1633 (@barton2526) - Change team whitelist delimiter to <> for CPID detection #1634 (@cyrossignol) - Change team whitelist separator to <> to accomodate more team names #1632 (@jamescowens) - Change Curl download speed type to support older environments #1640 (@cyrossignol) - Optimize logo SVGs used for tray icons #1638 (@cyrossignol) - Tweak consolidateunspent rpc function #1644 (@jamescowens) - ETTS and staking icon enhancements #1650 (@jamescowens) - Implement new transaction fees for block version 11 #1652 (@jamescowens) - Optimize in-memory storage of superblock data #1653 (@cyrossignol) - Miscellaneous superblock API improvements and housekeeping #1654 (@cyrossignol) - Update openssl to 1.1.1f compatibility #1660 (@jamescowens) - Optimize bdb to avoid synchronous flush of database #1659 (@jamescowens) - Add support for CPID input to "lifetime" RPC function #1668 (@cyrossignol) - Overhaul the contract handling system #1669 (@cyrossignol) - Make the autostart mainnet/testnet aware #1671 (@jamescowens) - Remove slashes from User Agent in peers tab #1674 (@div72) - Refactor contracts for polymorphic binary payloads #1676 (@cyrossignol) - Overhaul the beacon system #1678 (@cyrossignol) - Replace boost::optional<T&> with non-owning pointers #1680 (@cyrossignol) - Optimize proof-of-stake validation #1681 (@cyrossignol) - Updated Slack link #1683 (@NeuralMiner) - Update build-unix.md #1686 (@Quezacoatl1) - Replace deprecated QT methods #1693 (@Pythonix) - Made protocol.h more similar to bitcoin #1688 (@Pythonix) - Touch up some details for block version 11 #1697 (@cyrossignol) - More tweaks for block version 11 #1700 (@cyrossignol) - Finish the conversion to the BCLog class based logger #1699 (@jamescowens) - Move claim version transitional code in miner for proper signature #1712 (@cyrossignol) - doc: Update threads in coding.txt #1730 (@div72) - qt: Include QPainterPath in trafficgraphwidget.cpp #1733 (@div72) - doc: Update doc/build-unix.md #1731 (@div72) - gui: Show peers tab on connections icon click #1734 (@div72) - refactor: Change return type of IsMine to isminetype && move wallet files to wallet directory #1722 (@div72) - build: Updates boost to 1.73.0 for depends #1673 (@jamescowens) - doc: Update Unit Test Readme #1743 (@RoboticMind) - wallet: Change Assert To Error Message In kernel.cpp #1748 (@RoboticMind) - scraper: Shorten display representation of verification codes #1754 (@cyrossignol) - log: Change ".B." to Clear Message #1758 (@RoboticMind) - util: Fix braindamage in GetDefaultDataDir() #1737 (@jamescowens) - scraper: Improve scraper processing of beacon verifications #1760 (@jamescowens) - scraper: Add instrumentation to convergencereport #1763 (@jamescowens) - rpc: Improve rpc stress test script #1767 (@tunisiano187) - Generalize enum serialization #1770 (@cyrossignol) - scraper: Improve handling of ETags in http class and tweak verified beacon logic #1776 (@jamescowens) - scraper: Improve ProcessNetworkWideFromProjectStats and other tweaks #1778 (@jamescowens) - researcher: Automate beacon advertisement for renewals only #1781 (@cyrossignol) - gui: Tweak behavior of beacon page in researcher wizard #1784 (@cyrossignol) - Prepare for block version 11 hard-fork on testnet #1787 (@cyrossignol) - scraper: Modify UpdateVerifiedBeaconsFromConsensus #1791 (@jamescowens) - gui: Optimize OverviewPage::updateTransactions() #1794 (@jamescowens) - ci: Adopt ci changes from Bitcoin #1795 (@div72) - consensus: switch snapshot accrual calculation to integer arithmetic #1799 (@cyrossignol) - voting: Overhaul the voting system #1809 (@cyrossignol) - contract: Optimize contract replay after chain reorganization #1815 (@cyrossignol) - contract: Reimplement transaction messages as contracts #1816 (@cyrossignol) - staking: Sign claim contracts with coinstake transaction #1817 (@cyrossignol) - gui: Change research wizard text #1820 (@div72) - net: Update protocol version and clean up net messaging #1824 (@cyrossignol) - rpc, wallet: Corrections to GetAmounts #1825 (@jamescowens) - gui: Tweak some minor researcher wizard details #1830 (@cyrossignol) - gui: Change GetEstimatedStakingFrequency text #1836 (@jamescowens) - scraper: Scraper global statistics cache optimization #1837 (@jamescowens) - doc: Update Vulnerability Response Process #1843 (@RoboticMind) - scraper: Optimization of manifest and parts sharing between ConvergedScraperStatsCache, mapManifest, and mapParts #1851 (@jamescowens) - consensus: Update Checkpoints #1855 (@barton2526) - docs: Update docs to build off master #1856 (@barton2526) - gui: Fix and improve GUI combo box styles #1858 (@cyrossignol) - build: Tweak Gridcoin installer for Fern release #1863 (@jamescowens) Removed - Remove old research age checks (rebase #1365) #1572 (@cyrossignol) - Remove PrimaryCPID check from diagnostics dialog #1586 (@cyrossignol) - Remove missed label for PrimaryCPID from diagnostics #1588 (@cyrossignol) - Remove legacy quorum messaging system (@neural network) #1589 (@cyrossignol) - Remove old remnants of legacy smart contract experiments #1594 (@cyrossignol) - Remove block nonce for version 11 #1622 (@cyrossignol) - Delete obsolete contrib/Installer and Upgrader directories #1623 (@jamescowens) - Remove redundant LoadAdminMessages() calls #1625 (@cyrossignol) - Remove some legacy informational RPC commands #1658 (@cyrossignol) - Remove informational magnitude field from binary claims #1661 (@cyrossignol) - Remove fDebug3,4, and net and convert to BCLog::LogFlags #1663 (@jamescowens) - Remove qt5.7.1 depends support build System #1665 (@iFoggz) - Remove unused jQuery library #1679 (@cyrossignol) - Remove unused NetworkTimer() function and global state #1701 (@cyrossignol) - Refactor claim context objects into contracts #1704 (@cyrossignol) - Clean old assets up #1718 (@div72) - Remove legacy "rain" RPC (not by-project rain) #1742 (@cyrossignol) - Temporarily disable voting system on testnet #1769 (@cyrossignol) - gui: Remove legacy GUI transaction description for contracts #1772 (@cyrossignol) - gui: Remove transaction fee setting #1780 (@cyrossignol) - trivial: Cleanup unused legacy functions #1793 (@cyrossignol) - mining, rpc: Remove kernel-diff-best and kernel-diff-sum #1796 (@jamescowens) - refactor: Remove libs subdirectory #1802 (@div72) - scraper: cleanup unused/unnecessary functions #1803 (@jamescowens) - gui: Remove useless "Detach databases at shutdown" #1810 (@jamescowens) - test: Remove testnet condition for standard transactions #1814 (@cyrossignol) - consensus: Remove transitional testnet code #1854 (@cyrossignol) Fixed - Fix "Owed" amount in output of "magnitude" RPC method #1569 (@cyrossignol) - Add support for paths with special characters on Windows #1571 (@cyrossignol) - Fix lingering peers.dat temp files and clean up remaining paths #1582 (@cyrossignol) - Fix incorrect beacon length warning in GUI transaction list #1585 (@cyrossignol) - Fix default config file line endings on Windows #1587 (@cyrossignol) - Reenable Travis builds for MacOS #1591 (@jamescowens) - Correct peer detail info background color #1593 (@jamescowens) - Fix exception in debug3 mode #1598 (@cyrossignol) - Fix deadlock in "getmininginfo" RPC function #1596 (@cyrossignol) - Fix accuracy of statistics in "network" RPC output #1602 (@cyrossignol) - Fix heights for quorum vote weight calculations #1604 (@cyrossignol) - Fix deadlock in log archiver when rename fails #1607 (@cyrossignol) - Fix a spurious segmentation fault during client load on Windows with fast CPUs #1608 (@jamescowens) - Fix lock order debugging and potential deadlocks #1612 (@jamescowens) - Add dependencies #1613 (@Scalextrix) - Fix std namespace pollution #1617 (@denravonska) - Add missing condition for newbie accrual computer #1618 (@cyrossignol) - Track first reward blocks in research accounts #1619 (@cyrossignol) - Fix lingering beacon warning after advertisement #1627 (@cyrossignol) - Fix accrual calculation for new, zero-magnitude CPIDs #1636 (@cyrossignol) - Fix diagnostics, add ETTS test, fix tooltipcolor, add missing lock, and add email=investor check #1647 (@jamescowens) - Fix help message of two RPC methods #1656 (@div72) - Fix legacy accrual for newbie with non-zero past reward #1667 (@cyrossignol) - Fix GUI autostart on Windows for paths with wide characters #1670 (@cyrossignol) - Qualify boost bind placeholders with their full namespace #1672 (@Ponce) - Fix suffix when copying txids #1677 (@div72) - Unnecessary if-statement removed #1685 (@Pythonix) - Fix consolidatemsunspent Help Message #1687 (@Pythonix) - Fix gettransaction help message #1691 (@Pythonix) - Fix GetNewMint To Look for Stakes #1692 (@RoboticMind) - Suppress deprecated copy warnings for Qt with GCC 9+ #1702 (@cyrossignol) - Fix exclusion error on stats processing and misplaced ENDLOCK logging entry #1710 (@jamescowens) - Removed unnecessary comparison #1708 (@Pythonix) - Fixed typo #1707 (@Pythonix) - Fix out-of-bounds exception for peers tab version slashes #1713 (@cyrossignol) - Fix transition for v1 superblocks when reorganizing #1714 (@cyrossignol) - Touch up transition to version 2 transactions #1715 (@cyrossignol) - Avoid mutating transactions in ConnectBlock() #1716 (@cyrossignol) - Skip beacon advertisement when already pending #1726 (@cyrossignol) - Fix Windows cross-compilation in newer environments #1728 (@cyrossignol) - Fix out-of-bounds access in IsMineInner() #1736 (@cyrossignol) - Fix a couple of block version 11 issues #1738 (@cyrossignol) - Fix null pointer dereference in GUI researcher model #1741 (@cyrossignol) - accrual: Reset research accounts when rebuilding accrual snapshots #1745 (@cyrossignol) - scraper: Correct update for verified beacons #1747 (@jamescowens) - accrual: Refactor tally initialization for snapshot rebuild #1749 (@cyrossignol) - rpc: Fix "cpid" field in "beaconconvergence" RPC output #1750 (@cyrossignol) - accrual: Fix snapshot accrual superblock state transitions #1752 (@cyrossignol) - scraper: Correct stale verified beacon logic #1753 (@jamescowens) - rpc: Correct possible divide by zero in getblockstats #1755 (@jamescowens) - gui: Fix issues with researcher wizard flow #1756 (@cyrossignol) - wallet: Stop Error When Starting From Zero #1759 (@RoboticMind) - Don't count empty email as explicit investor #1761 (@cyrossignol) - accrual: Fix snapshot accrual superblock state transitions #1764 (@cyrossignol) - rpc: Cleanup Help Message and Fix Typo #1771 (@RoboticMind) - scraper: Fix scraper etag header case sensitivity #1773 (@cyrossignol) - consensus: Use explicit time to check if superblock needed #1774 (@cyrossignol) - gui: Fix scroll area dark theme styles #1785 (@cyrossignol) - rpc, gui: Fix three divide by zero possibilities #1789 (@jamescowens) - rpc: Fix balance pre-check in "rainbymagnitude" RPC #1792 (@cyrossignol) - accrual: Fix outdated comment and correct grammar #1800 (@RoboticMind) - gui: Fix stuck cursor on labels #1801 (@div72) - beacon: Fix research wizard beacon renewal status #1805 (@cyrossignol) - gui: Fix translations for port numbers #1818 (@cyrossignol) - util: Create parent directory #1821 (@div72) - mining: Fix coinstake/claim signature order #1828 (@cyrossignol) - voting: Remove double increment in loop #1831 (@cyrossignol) - neuralnet, scraper: Fix compilation with gcc5 and older libcurl #1832 (@a123b) - wallet: Fix smallest coin selection for contracts #1841 (@cyrossignol) - gui: Fix display of polls with no votes yet #1844 (@cyrossignol) - gui: add indentation to diagnostic status bar labels #1849 (@jamescowens) - voting, gui: Fix formatting and alignment of vote shares and percent #1850 (@jamescowens) - wallet, rpc: Fix for self-transactions in listtransactions #1852 (@jamescowens) - accrual: Clear any accrual snapshots when syncing from pre-v11 #1853 (@cyrossignol) - accrual: Fix reset of accrual directory if starting sync below research age height #1857 (@jamescowens) - gui: Fix researcher wizard layout on macOS with native theme #1860 (@cyrossignol)
This PR rewrites the voting system. The changes improve the performance, reliability, security, and maintainability of the voting features in Gridcoin. I replaced the legacy voting functions with a new set of classes for creating and consuming poll and vote contracts.
These contract types now include contract payload implementations for binary serialization of contract data in the block version 11 protocol. This reduces the size and serialization overhead of these contracts. Legacy XML-like string contracts contain several unnecessary fields that bloat voting transactions. I stripped the unnecessary fields when designing the binary formats.
To continue to move away from legacy
AppCache
state, I added anIContractHandler
implementation that manages poll and vote contract data. Unlike theAppCache
—which stored whole contracts in memory—thePollRegistry
class only maintains "bookmarks" for polls and votes used to look up the contracts from disk when generating a poll result. This reduces memory usage and improves synchronization speed.I removed the the deprecated "magnitude", "CPID count", and "participant count" poll weight types (#433). The GUI and RPCs do not support creation of polls with these options anymore, and the protocol will reject polls that declare these weight types after the switch to block version 11.
The new voting classes no longer bury RPC exceptions in the domain logic (#1370). Input validation for polls and votes will throw agnostic
VotingError
exceptions. I increased the scope of the local validation in the set of "builder" classes that construct the contracts from user input and wallet state (#131, #219, #940, #947, #1025).Legacy voting code relied on input of poll title and answer strings for association and danced around the problems by replacing spaces in these with underscores, and it failed to account for other issues with Unicode and duplicates. With this PR, polls and votes use canonical IDs to associate items: votes refer to polls by TXID (transaction hash) and to choices by offset (as defined in the original poll). This deprecates RPC parameters that accept title and answer label strings and we may remove this behavior in the future (#527). Voters are encouraged to use the
votebyid
RPC function instead ofvote
to cast a vote by poll ID and answer ID. Voting contracts now formally expect strings with UTF-8 encoding.I added the GUI wallet unlock prompt to the voting dialogs and a transaction record category for polls. The category affixes the same icon to poll transactions as it does for votes now. Before, polls appeared as blank "send" transactions which seemed confusing.
Although the voting GUI can really benefit from some design changes, I did not modify the GUI except as described above. This PR focuses mostly on core behavior. We will likely revisit the voting GUI when implementing the redesign as a whole.