Mf/add dao coin limit order id field #327
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mattfoley8
commented
Apr 13, 2022
lazynina
requested changes
Apr 13, 2022
mattfoley8
commented
Apr 13, 2022
lazynina
approved these changes
Apr 14, 2022
piratekev
added a commit
to TheBitgram/core
that referenced
this pull request
May 18, 2022
* Use nil blockhashex on mempool transactions (deso-protocol#140) * Add a list of nodes maintained by the core developer team (deso-protocol#145) Co-authored-by: Erwin Willems <github@reflexlight.nl> Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> * Limit mempool txn size to half of block size * Fix BTC buy check * Increase logging on BlockCypherCheckBitcoinDoubleSpend * Simpler derived keys (deso-protocol#148) * Simplify * Fix Co-authored-by: p <p> * Added additional nodes (deso-protocol#149) * add new node (deso-protocol#152) * Missing comma on newest node (deso-protocol#157) * Add DeversoNode (deso-protocol#156) * add double spend detection source in error message (deso-protocol#159) * add double spend detection source in error message * Add txn hash to double-spend error logging Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Add support for nft transfer notifications (deso-protocol#158) * Add support for nft transfer notifications * add trailing comma in nodes.go * Move vendored libraries to separate repos (deso-protocol#160) * [stable] Release 1.2.8 * update GetPostsPaginatedForPublicKeyOrderedByTimestamp to take in nftsRequired boolean (deso-protocol#161) * Adding node (deso-protocol#163) * Split block view and tests (deso-protocol#164) * expose the prefix for the index from public key to deso balance (deso-protocol#168) * expose the prefix for the index from public key to deso balance * expose prefix for creator deso locked * Factor the double-spend check into its own function (deso-protocol#165) * Factor the double-spend check into its own function * return boolean indicating if error is from double spend detection or not (deso-protocol#167) * return boolean indicating if error is from double spend detection or not * flip return values order Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Ln/fix minor disconnect bid issue (deso-protocol#169) * [stable] Release 1.2.9 * Buy Now NFTs with Buy Now Price (FORKING CHANGE) (deso-protocol#166) * Refactor connectAcceptNFTBid to move logic that handles transferring the NFT, computing royalties and payouts, and executing payouts so it can be reused in connectNFTBid to support Buy now NFTs * update disconnect logic for nft bid to account for buy now NFTs * update tests and disconnect logic for NFT bids * finish buy now test and add more comments around test * Add NFTBidCreatorPublicKey, NFTBidBidderPublicKey, NFTBidCreatorRoyaltyNanos to UtxoOperation for NFTBid transactions * remove unused BidderInputs from NFTBidMetadata, add comments above IsBuyNowKey * update notifications for NFT bid on buy now nft * remove commented out line * clean up comments * add block height to trigger Buy Now NFTs * fix tests involving buy now nfts * address RPH feedback * address a lot of DH feedback * Update to have a separate price for Buy Now NFTs instead of using MinBidAmountNanos * force Buy Now price to be greater than min bid amount * check in current progress on tests * finish out testing of Buy now NFT with separate buy now price * submitting a bid on serial number 0 will not trigger a buy now operation. You must specifically bid on a serial number for a buy now to occur * resolve merge conflicts with main * resolve conflicts in block_view_nft.go * set rosetta fields appropriate in helpConnectNFTSold * remove rosetta fields from initial construction * diamondhands review, clean up, fix a couple minor bugs * Moved the _helpConnectNFTSold block to closer to where it was previously to make reviewing easier. * Caught an error in CreateAcceptNFTBid that wasn't being caught before. * Fixed the ordering of adding to the utxoOperations list in _helpConnectNFTSold. The ops were being added in the wrong order before. In particular, we were doing: - _connectBasicTransfer - Spending the BidderInputs - Adding the royalties But the UtxoOps were `[bidder inputs, add royalties, connect basic transfer]`, when they should have been `[connect basic transfer, bidder inputs, add royalties]`, i.e. same order as the operations we performed. * Fixed some comments here or there Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * DAO coins (deso-protocol#174) * Refactor connectAcceptNFTBid to move logic that handles transferring the NFT, computing royalties and payouts, and executing payouts so it can be reused in connectNFTBid to support Buy now NFTs * update disconnect logic for nft bid to account for buy now NFTs * update tests and disconnect logic for NFT bids * finish buy now test and add more comments around test * Add NFTBidCreatorPublicKey, NFTBidBidderPublicKey, NFTBidCreatorRoyaltyNanos to UtxoOperation for NFTBid transactions * remove unused BidderInputs from NFTBidMetadata, add comments above IsBuyNowKey * update notifications for NFT bid on buy now nft * remove commented out line * clean up comments * add block height to trigger Buy Now NFTs * fix tests involving buy now nfts * address RPH feedback * address a lot of DH feedback * Update to have a separate price for Buy Now NFTs instead of using MinBidAmountNanos * force Buy Now price to be greater than min bid amount * check in current progress on tests * finish out testing of Buy now NFT with separate buy now price * submitting a bid on serial number 0 will not trigger a buy now operation. You must specifically bid on a serial number for a buy now to occur * resolve merge conflicts with main * resolve conflicts in block_view_nft.go * set rosetta fields appropriate in helpConnectNFTSold * remove rosetta fields from initial construction * Initial check-in of DAO Coin PR * add migration file * add more comments and make some code a little cleaner * address piotr feedback * address more Piotr feedback and add network tests * update HasPurchased in GetHoldings and GetHolders * fix indent on OperationTypeDAOCoinTransfer * add transfer restrictions to DAO coins * update postgress accordingly{ * update postgres handling * fix rule error capitalization * diamondhands review, clean up, fix a couple minor bugs * Moved the _helpConnectNFTSold block to closer to where it was previously to make reviewing easier. * Caught an error in CreateAcceptNFTBid that wasn't being caught before. * Fixed the ordering of adding to the utxoOperations list in _helpConnectNFTSold. The ops were being added in the wrong order before. In particular, we were doing: - _connectBasicTransfer - Spending the BidderInputs - Adding the royalties But the UtxoOps were `[bidder inputs, add royalties, connect basic transfer]`, when they should have been `[connect basic transfer, bidder inputs, add royalties]`, i.e. same order as the operations we performed. * Fixed some comments here or there * Fix merge conflict with test * Merbe _connectCreatorCoinTransfer and _connectDAOCoinTransfer * [REQUIRES RESYNC] Cleanup and rename CoinEntry -> CreatorCoinEntry * Move HelpConnectCoinTransfer * Add missing import * [REQUIRES RESYNC] Switch to uint256 for coin balances * In txns: - DAOCoinMetadata * CoinsToMintNanos * CoinsToBurnNanos - DAOCoinTransferMetadata * DAOCoinToTransferNanos In db structs: - CoinEntry * CoinsInCirculation * Didn't need CoinWatermarkNanos because CreatorCoins don't go over uint64 - BalanceEntry * BalanceNanos All tests passing * Add overflow tests for DAO coins * Nit: Add error logging for balance lookup Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Ln/nft splits (deso-protocol#173) * Refactor connectAcceptNFTBid to move logic that handles transferring the NFT, computing royalties and payouts, and executing payouts so it can be reused in connectNFTBid to support Buy now NFTs * update disconnect logic for nft bid to account for buy now NFTs * update tests and disconnect logic for NFT bids * finish buy now test and add more comments around test * Add NFTBidCreatorPublicKey, NFTBidBidderPublicKey, NFTBidCreatorRoyaltyNanos to UtxoOperation for NFTBid transactions * remove unused BidderInputs from NFTBidMetadata, add comments above IsBuyNowKey * update notifications for NFT bid on buy now nft * remove commented out line * clean up comments * add block height to trigger Buy Now NFTs * fix tests involving buy now nfts * address RPH feedback * address a lot of DH feedback * Update to have a separate price for Buy Now NFTs instead of using MinBidAmountNanos * force Buy Now price to be greater than min bid amount * check in current progress on tests * finish out testing of Buy now NFT with separate buy now price * submitting a bid on serial number 0 will not trigger a buy now operation. You must specifically bid on a serial number for a buy now to occur * resolve merge conflicts with main * resolve conflicts in block_view_nft.go * check in current progress on NFT splits * start test of nft splits * add IsDeleted function to pkid entries * add more error case testing * add comments, resolve TODOs, and check for overflow * set rosetta fields appropriate in helpConnectNFTSold * remove rosetta fields from initial construction * refactor construction of additional royalties to reduce code duplication * go fmt project * Review splits. Ready to ship! * Made custom deterministic serializers rather than using gob * Changed interface to use public key, hiding PKID * Added SwapIdentity to the test on splits. Looks good * Fixed a minor overflow bug * TODO: Add a test for overflows * add more checks for overflow and add more testing for overflow * add one more test on overflow for good measure Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Fix bug introduced by NFT splits (deso-protocol#176) * Add Tunel node (deso-protocol#177) * Check CreatorPKID instead of HODLerPKID when iterating over utxoview (deso-protocol#178) * Move blockheights into DeSoParams and set values accordingly (deso-protocol#171) * Move blockheights into DeSoParams and set values accordingly * fix test * remove unnecessary update to set blockheight * resolve conflicts with main * fix alignment and remove original constants for block heights * go fmt constants.go * fix BuyNowAndNFTSplitsBlockHeight usage in extractAdditionalRoyaltyMap * use params.DAOCoinBlockHeight * use params.BuyNowAndNFTSplitsBlockHeight * Embed fork heights in a struct * Make testnet heights equal to previous values and fix broken test Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Messages V3 (deso-protocol#182) * Finish connect/disconnect logic * Add requests * Add Party Entry, Start Connect/Disconnect * Finish connecting party messages; all core logic done * Small rename * Fix messaging key inconsistencies; general clean * Cleanup code; Add tests for messaging keys * Fix weird indents * Add initial tests for message parties * Finish testing * Thoroughly tested and some more cleanup * Add support for get/send backend endpoints * Update backend endpoint * Add block height * Lang * Finish core & add tests * Finish testing messaging keys; hard part done, now onto private message tests * Cleanup test comments * Fix private message connect/disconnect; cleanup * Add more tests * Fix setters/getters for message entries * All done & tested * Final cleanup * Add network tests * Fix isRecipient * Simplify messaging key entry * Simplify messaging key entry (deso-protocol#181) * Unencrypted groups support * Fork heights * Clean version code * Nits * Upgrade some comments * Clean up messages v3 - Fixed isDeleted bug in the fetching of group messages - Rename everything to be group-oriented: * MessagingKey -> MessagingGroupKey * MessagingKeyEntry -> MessagingGroupEntry * recipient -> member * MessagingRecipient -> MessagingGroupMember - Refactor some db stuff. Mainly added GroupOwnerPublicKey to the MessagingGroupMember, and cleaned up the back-reference there. * Test fix * Final passerinio * Good luck pass; and prefix fix * Unfix prefix Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Fix instances where a pointer is taken of a map key iterator (deso-protocol#183) * Fix another iterator pointer issue (deso-protocol#184) * Fix stubborn NFT splits bug due to reordering of royalties in map (deso-protocol#188) * [stable] Release 2.0.0 (deso-protocol#190) * Set block height triggers for DAO Coins, NFTs v2, and Messages v3 * [stable] Release 2.0.1 * Fix sql statements for DAOCoin/Nft (deso-protocol#193) * Add new constant to constants file to represent the NodeSource map key in Extra Data (deso-protocol#191) * Adding entre's node. (deso-protocol#179) * nft and dao coin notifications after fork (deso-protocol#185) * add support for DAO coin and nft royalty notifications * updates to better support notifications on the frontend * fix issue with pointers in for loops * merge in main and copy iterator * fix creator pub key * Fix creator public key in nft bid * skip affected pub key if royalty amount is 0 * simplify owner public key in NFT bid case * simplify creator public key in Accept NFT Bid case * add deso royalty map to utxo * make the NFT bid case mirror accept NFT bid * [stable] Release 2.0.2 * Update Postgres handling of DAO Coins and NFTs (deso-protocol#194) * [stable] Release 2.0.3 * Add a SQL adapter for BlockHash that makes it work with bun (deso-protocol#204) This will allow us to upgrade to Postgres v14 bu using the bun library rather than the super old go-pg library, which is still stuck at v10 * fix DAO coin postgres issues (deso-protocol#208) * [stable] Release 2.0.4 (deso-protocol#209) * create maps for additional royalties when converting to/from PGPostEntry before copying values into it (deso-protocol#212) * create maps for additional royalties when converting to/from PGPostEntry before copying values into it * Add debugging to error path * Update encoding of pkid in AdditionalNFTRoyaltiesToCoinsBasisPoints * Add group messaging support * Add messaging group tables * Fix small bug in down of migration Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Return error if error during txindex db update (deso-protocol#220) * Fix Issue with ParentStakeID set when modifying Post (deso-protocol#222) * move ComputeMetadata to a switch-case and glog errors for Like and SubmitPost Txns * don't return an error from ComputeTransactionMetadata, just glog it * Create accept bid for buynow when create bid in postgres (deso-protocol#226) * Create accept bid for buynow when create bid in postgres * Update lib/postgres.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * add AcceptedBlockHeight to NFTBidEntry (REQUIRES RESYNC) (deso-protocol#224) * add AcceptedBlockHeight to NFTBidEntry * Set Accepted on PGNFTBid is AcceptedBlockHeight is not nil * only copy AcceptedBlockHeight if not nil * add txindex metadata for Burns and Accept NFT Transfers (deso-protocol#228) * Update nodes.go (deso-protocol#202) * Update nodes.go * Update nodes.go * Add upcoming overclout node (deso-protocol#219) * Add upcoming overclout node * Update lib/nodes.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Fix get-all-messaging-keys & add base point test (deso-protocol#217) * add remaining UtxoTypes to UtxoType.String() (deso-protocol#229) * [stable] Release 2.0.6 (deso-protocol#231) * ch pgBidNft is not nil (deso-protocol#232) * [stable] Release 2.0.7 (deso-protocol#233) * Ln/spending limits (deso-protocol#216) * Fix profile vulnerability. (#1) * lower height * Add transaction spending limit for derived keys save current progress on derived key spending limit add more testing, remove usage of pointers in limit map keys * separate SET and TRACK block heights for derived key transaction spending limits * remove weird test * address Piotr feedback - encoders/decoders for map keys, add comments, refactor inline functions * go fmt project * add TransactionSpendingLimitTracker and Memo to PGDerivedKey * updates to converting DAO/NFT/CC operations to string for backend usage * compare operation limit keys by hex encoding * add more tests * diamondhands edits * merge in main * Add ZeroBlockHash and use ZeroBlockHash and ZeroPKID in map keys * diamondhands touchups * set regtest forkheights to 0 * add test that derived key can sign its own transactions and spend its own money * Address fixmes * rebuild * add test case for minting NFT * reduce nested if statements in block_view.go * Update lib/block_view_types.go Co-authored-by: redpartyhat <81658108+redpartyhat@users.noreply.github.com> Co-authored-by: maebeam <maebeam0@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Extra Data on Posts, NFTs, Messages, Messaging Groups, and Derived Keys (deso-protocol#213) * Add ProfileExtraData to ProfileEntry struct * add test to ensure extra data comes through. update calls to CreateUpdateProfileTxn, add postgres support * Update lib/block_view_profile_test.go * Review and cleanup * Add one more fixme * update profile test for extra data * ExtraData on NFT / Message / GroupMessagingKey / DerivedKey entries (deso-protocol#221) * Return error if error during txindex db update (deso-protocol#220) * Add extra data to NFT, Message, GroupMessagingKey, and DerivedKey entries Return error if error during txindex db update (deso-protocol#220) add tests for adding extra data on NFT, private message, messaging group, and derived key entries add comment on extra data * Fix Issue with ParentStakeID set when modifying Post (deso-protocol#222) * move ComputeMetadata to a switch-case and glog errors for Like and SubmitPost Txns * don't return an error from ComputeTransactionMetadata, just glog it * Create accept bid for buynow when create bid in postgres (deso-protocol#226) * Create accept bid for buynow when create bid in postgres * Update lib/postgres.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * add AcceptedBlockHeight to NFTBidEntry (REQUIRES RESYNC) (deso-protocol#224) * add AcceptedBlockHeight to NFTBidEntry * Set Accepted on PGNFTBid is AcceptedBlockHeight is not nil * only copy AcceptedBlockHeight if not nil * add txindex metadata for Burns and Accept NFT Transfers (deso-protocol#228) * Update nodes.go (deso-protocol#202) * Update nodes.go * Update nodes.go * Add upcoming overclout node (deso-protocol#219) * Add upcoming overclout node * Update lib/nodes.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Fix get-all-messaging-keys & add base point test (deso-protocol#217) * add remaining UtxoTypes to UtxoType.String() (deso-protocol#229) * diamondhands touchups with some FIXMEs for @lazynina * delete fixmes about deleting protected keys * add test for encoding/decoding message entries for backwards compatibility * add test for encoding/decoding messaging group entries for backwards compatibility Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> Co-authored-by: elmasapp <96642041+elmasapp@users.noreply.github.com> Co-authored-by: Don Hardman <59067742+donhardman@users.noreply.github.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Update lib/blockchain.go Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> Co-authored-by: elmasapp <96642041+elmasapp@users.noreply.github.com> Co-authored-by: Don Hardman <59067742+donhardman@users.noreply.github.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com> * Gofmt the whole repo * simplify extra data handling in CreateAuthorizeDerivedKey (deso-protocol#235) * operationtype typo in disconnect (deso-protocol#238) * Fix postgres down migrations. (deso-protocol#276) * Make derived key txn construction accept a spending limit hex, not object (deso-protocol#297) * Make derived key txn construction accept a spending limit hex, not object * Update lib/block_view_derived_key.go * add exists declaration back. Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Use NewPKID to get bidder PKID if bidder's profile is not found (deso-protocol#262) * Feature/add dao coin limit order transaction (deso-protocol#307) * Add DAO Coin Limit Order network skeleton. (deso-protocol#239) * Add DAO Coin Limit Order constants. (deso-protocol#240) * Add DAO Coin Limit Order mempool skeleton. (deso-protocol#243) * Add DAO Coin Limit Order mempool skeleton. * Restore removed imports. * Side-step unused var error. * Add DAO Coin Limit Order blockchain skeleton. (deso-protocol#242) * Add DAO Coin Limit Order blockchain skeleton. * Restore removed imports. * Add DAO Coin Limit Order block view skeleton. (deso-protocol#241) * Add DAO Coin Limit Order block view skeleton. * Restore removed import. * Side-step unused var error. * Add DAO Coin Limit Order block view flush skeleton. (deso-protocol#244) * Add DAO coin limit order db prefixes. (deso-protocol#245) * Mf/add dao coin limit order key builders (deso-protocol#246) * Add DAO coin limit order key builders. * Use uint32s for enums. * Remove unnecessary line. * Mf/add dao coin limit order db utils (deso-protocol#247) * Add DAO coin limit order db utils. * Add FromBytes skeleton method. * Fix FromBytes operation happens in-place. * Mf/add dao coin limit order postgres skeleton (deso-protocol#248) * Add DAO coin limit order postgres skeleton. * Fix spacing/tabs. * Fix method call capitalization. * Convert uint256 types to strings. * Add DAO coin limit order network metadata. (deso-protocol#249) * Revert DAO coin limit order enums to uint8s. (deso-protocol#250) * Add DAO coin limit order connect transaction logic. (deso-protocol#259) * Mf/add dao coin limit order txn testing (deso-protocol#263) * Switch DAO coin order limit quantity to pointer. (deso-protocol#265) * Add more DAO coin limit order testing scenarios. (deso-protocol#267) * Convert DAO coin limit order price from BigFloat to Uint256. (deso-protocol#266) * Convert DAO coin price to uint256. * Complete conversion of DAO limit order price to uint256. * Use Lt, Gt instead of Cmp. * Start inverting price comparison logic. * add function to calculate total cost of an order (deso-protocol#268) * add function to calculate total cost of an order * Remove Conditional from DBGetMatchingDAOCoinBidOrders that didn't apply, Fix IsBetterAskThan and IsBetterBidThan (deso-protocol#269) * add check that order is better than last seen order when getting next orders to fill, fix txn metadata for m1's txn in test * use total cost calculating function to accurately get total cost of orders in blockchain.go, set prefixKey correctly for DBGetMatchingDAOCoinAskOrders Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Mf/track dao coin limit order utxos (deso-protocol#270) * Track DAO coin limit order UTXOs. * Add disconnect logic. * Fix failing test. * Update lib/block_view_dao_coin_limit_order.go * Add DAO coin limit order postgres structs. (deso-protocol#271) * Add DAO coin limit order postgres structs. * Update dao coin limit order postgres migration. * Fill out LeftPaddedHexToUint256. * Add DAO coin limit order postgres tests. (deso-protocol#273) * Remove logging before flags error. (deso-protocol#274) * Fix DAO coin down migration. (deso-protocol#275) * Fix some postgres testing issues. (deso-protocol#277) * Add additional DAO coin limit order test and resolve bugs it surfaces. (deso-protocol#278) * Add additional DAO coin limit order testing. * Refactor block view to use db adapter for DAO coin limit orders. * Set up multi-order test. * Give m0 more initial in testing. * Fix failing txn spend amt test. * Fix updating order's quantity bug. * Add helper function to query for all orders. * Fix by transactor PKID prefix bug. * Mf/better dao coin limit order var naming (deso-protocol#279) * Refactor DAO coin limit order variable naming. * Rename order to matchingOrder. * Test DAO coin limit order total cost doesnt overflow validation. (deso-protocol#280) * Mf/allow dao coin limit order cancellations (deso-protocol#281) * Allow DAO coin limit order cancellations. * Fix failing postgres test. * Add more comments. * Add failing test to show no transactorPKID in index bug. * Update helper utils to new order schema. * Checkpoint. Store progress updating connect logic. * Another checkpoint. Stash progress in connect logic. * Commit checkpoint. Updated all of block view. * Start rebuilding tests. * Checkpoint. Continue updating tests. * Add back endpoint to query open orders by transactor PKID. * Add function to get all limit orders for a DAO coin pair. * Clean up postgres. * Convert price from uint256 to big.Float. * Add uint256 ScaledPrice field. * Continue turning back-on more tests. * Fix more tests. * Checkpoint. Continue fighting tests. * Continue fixing tests. * Fix last two tests. * Revised a lot of things, but mainly _connect logic. Let's have a meeting to discuss. * Convert PKID vars to PublicKeys. * Start fixing tests. * Fix tests by removing reflect.DeepEqual. * Switch buying and selling DAO coins back to PKIDs. * Fix a few more failing tests. * Must compare transactor buying qty to matching selling qty. * Continue fixing tests. * Get cancellation tests passing. * Only update orders within the same block height. * Fix small postgres bug. * Ln/dao coin limit order txindex (deso-protocol#283) * initial check in for txindex for dao coin limit orders * Use new struct to track fulfilled orders * update naming convention * Mf/add safe math uint256 utils (deso-protocol#285) * Add SafeMathUint256 utils. * Convert some DAO coin limit order math to safe math. * Save me some whitespace. * Mf/add safe math uint64 utils (deso-protocol#287) * Add SafeMathUint256 utils. * Convert some DAO coin limit order math to safe math. * Save me some whitespace. * Add SafeMathUint64 utils. * Update DAOCoinLimitOrderMetadata type so it can be serialized to json (deso-protocol#286) * Update DAOCoinLimitOrderMetadata type so it can be serialized to json * Re-autoformat comments * Address feedback on sorting slices * Address feedback in connect transaction function * Mf/add dao coin transfer restriction validations (deso-protocol#288) * Add DAO coin limit order transfer restriction validations. * Make a bit cleaner. * Keep full error message. * Add UTXO view API DAO coin limit order getters. (deso-protocol#290) * Add UTXO view API DAO coin limit order getters. * Make merging database and UTXO values more consistent. * Only compute order map key once. * Ln/dao coin limit order disconnect (deso-protocol#289) * Add temporary postgres DAO coin limit order solution. (deso-protocol#292) * Initial check in for spending limits on DAO Coin Limit Orders (deso-protocol#293) * Initial check in for spending limits on DAO Coin Limit Orders * add comments, rename key to DAOCoinLimitOrderLimitKey, initialize map in copy, add tests for spending limit on dao coin limit orders * Add additional DAO coin limit order tests. (deso-protocol#291) * Ln/dao coin limit order cancellation validations (deso-protocol#294) * Add additional DAO coin limit order tests. * fix infinite loop with last seen order, fix validations around cancellations, update tests Co-authored-by: mattfoley8 <isaac@deso.org> * Address fixmes: add sanity checks and comments. Add isDeleted checks and new RuleErrors (deso-protocol#295) * Ln/dao coin limit order loop till filled (deso-protocol#296) * Loop until order is filled, fix db utils for getting next order, add more tests, fix copy DAOCoinLimitOrderEntry to not return error * leave comment in test for diamondhands * add balance checks in latest test * Diamondhands touch-ups * Add postgres CI whitelisted testing. (deso-protocol#298) * Add postgres CI whitelisted testing. * Run migrations before testing. * Map postgres port to localhost. * Empty commit to test caching dependencies. * Test migration rollbacks as well. * Add newline at EOF. * Mf/add additional dao coin order book tests (deso-protocol#299) * Add additional DAO coin order book tests. * Dont say bid/ask. * Add go-fmt to CI. (deso-protocol#300) * Add go-fmt to CI. * Fix failing gofmt error. * Address DH fixmes (deso-protocol#301) * Add another test. (deso-protocol#302) * Fix inputs from transactor to account for fee AND DESO spent purchasing DAO Coins, fix postgres, and add FeeNanos to txn metadata (deso-protocol#303) * Fix inputs from transactor to account for fee AND DESO spent purchasing DAO Coins, fix postgres, and add FeeNanos to txn metadata * Clone quantity to be safe * go fmt blockchain.go * always write utxo ops to badger (deso-protocol#304) * always write utxo ops to badger * run disconnect for pg * fix table name for dao coin limit order metadata * Add local postgres testing setup. (deso-protocol#305) * Make derived key txn construction accept a spending limit hex, not object (deso-protocol#297) (deso-protocol#306) * Make derived key txn construction accept a spending limit hex, not object * Update lib/block_view_derived_key.go * add exists declaration back. Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: diamondhands0 <81935176+diamondhands0@users.noreply.github.com> * Add fee nanos to the to bytes and from bytes logic for DAO Coin Limit Orders (deso-protocol#308) * Error if matching own order. (deso-protocol#309) * Make API getter functions public (deso-protocol#310) * Mf/add operation type column (deso-protocol#311) * Rename quantity to buy -> quantity to fill. * Checkpoint passing tests. * Get postgres tests working. * Add helper function and tests. * Continue adding tests for operation type quantity math. * Complete BID-BID calculation logic. * Build out ask-ask case. * Use named return values instead of a struct. * Allow for param specifying different transactor quantity to fill. * Implement logic in db_utils. * Refactor calculate test into own test. * Update blockchain.go w/ new quantity calculation. * Throw error if updating order w/ different operation type. * Update connection logic. * Fix failing derived key test. * Document off-by-one bug. * Add check on FeeNanos in DAO Coin Limit Order connect logic * Check for overflow before mutliplying fee nanos * revert changes to UpdateGlobalParams txn construction * Update w/ PR review feedback. * Make price calculators var naming a bit more consistent. * Remove DAO prefix from filled order struct qty fields. * Empty commit to trigger CI. * Fix off-by-one bug. * Fix floating point issues in tests, and implement fixed-point conversion scheme * Some touch-ups to the order matching logic. * Add a FIXME for an ultimate sanity-check in _connectDAOCoinLimitOrder * Add money printer prevention validation. * WIP - fix prevBalances, but sanity-check fails * Fix prevBalances bug and fix sanity-check after * Resolve merge conflicts. * Fix tests with a workaround on fee computation * diamondhands holy tidy pass * Fix failing postgres test. Co-authored-by: lazynina <lazynina84@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Add DAO coin limit order blockheight validation. (deso-protocol#314) * Add DAO coin limit order blockheight validation. * Empty commit to retrigger CI. * Set DAO coin limit order block height in derived key test. * Fix get next orders to fill qty check. (deso-protocol#315) * Add some cosmetic updates. (deso-protocol#316) * don't return error from functions that don't generate errors (deso-protocol#318) * TotalInput and TotalOutput should consider inputs and outputs to/from bidders and difference should equal fee (deso-protocol#317) * TotalInput and TotalOutput should consider inputs and outputs to/from bidders and difference should equal fee * Update lib/block_view_dao_coin_limit_order.go Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> * Update lib/block_view_dao_coin_limit_order.go Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> * Test submitting ASK order and matching existing BID. (deso-protocol#319) Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: iamsofonias <99746187+iamsofonias@users.noreply.github.com> Co-authored-by: diamondhands0 <81935176+diamondhands0@users.noreply.github.com> Co-authored-by: lazynina <lazynina84@gmail.com> * Mf/additional dao coin limit order testing (deso-protocol#324) * Only copy TransactionSpendingLimitTracker if non-nil, (tsl *TransactionSpendingLimit) Copy() should return nil if tsl is nil (deso-protocol#325) * Mf/add dao coin limit order id field (deso-protocol#327) * Start adding DAO coin limit order ID field. * Add OrderID field to connect logic cancellation. * Simplify MapKey now that we have an order uuid. * Fix failing postgres tests. * Fix failing derived key test. * Fix comments / spacing. * Add OrderID to FilledOrder. * Store copy of order in prev in case of rollback. * Add additional postgres indices. * Improve parity between Postgres and Badger result set ordering. * Address review feedback pt 1. * Only store metadata CancelOrderID in bytes if present. * Use maxHash as max block hash. * Mf/dao coin order metadata optional fields (deso-protocol#329) * DAO coin order metadata optional fields. * Encode optional fields with byte count prefix. * Clean up blockchain.go branching logic. * Clean up more branching logic. * Better English in comments. * Remove TransactorPKID from primary DAO coin order book index. (deso-protocol#331) * Remove TransactorPKID from primary DAO coin order book index. * Fix failing postgres test. * Fix intermittent badger CI failure. * Remove sorting by quantity. * Remove price and blockheight from by transactor index. * Mf/test swap identity and dao coin orders (deso-protocol#330) * Test swap identity and DAO coin orders. * Test moar swapping identity + order book cases. * Make postgres tests pass. * Update m0s PKID in tests. * Add test that owner can deauthorize derived key with new transaction spending limit (deso-protocol#326) * [TESTNET HARD FORK]: Update TESTNET block heights for DAOSwap hard fork (deso-protocol#332) * [TESTNET HARD FORK]: Update TESTNET block heights for DAOSwap hard fork The following hard forks are triggering on Testnet on Wed Apr 20 @ 9am ET: ExtraDataOnEntriesBlockHeight: uint32(304087), DerivedKeySetSpendingLimitsBlockHeight: uint32(304087), DerivedKeyTrackSpendingLimitsBlockHeight: uint32(304087), DAOCoinLimitOrderBlockHeight: uint32(304087), * Update DerivedKeyTrackSpendingLimitsBlockHeight * [stable] Release 2.1.0 (deso-protocol#333) * Mf/add unused bidder inputs test (deso-protocol#334) * [stable] Release 2.1.1 (deso-protocol#335) * Mf/additional bidder input tests (deso-protocol#339) * Add tests for metadata FeeNanos. (deso-protocol#337) * Add tests for metadata FeeNanos. * Revert change to connect logic. * Mf/bidder input edge case testing (deso-protocol#338) * Bidder input edge case testing. * Add DAO<-->DAO coin test. * Fix test for including spent UTXOs. * Try increasing fee rate and retesting. * Unused UTXOs get refunded. * Use m2's public key. * Mf/add limit order metadata order type field (deso-protocol#342) * Add limit order metadata OrderType field. * Update postgres moar. * Dynamically determine order book transactor fees. * Add helper function for calculating FeeNanos. * Use uint8s for enums. * Add txn metadata validations. * Small refactor as vars used only once. * Add ImmediateOrCancel order type. * Allow zero price on IOC and FOK orders. * Allow user to specify exchange rate on IOC and FOK orders. * Add tests for market ASK orders. * Store OrderEntry.OrderType in the database. * diamondhands touchups * Require OrderType must be specified. No default value. * Refactor IsMarketOrder util function location. * Validate txn metadata. * Rename OrderType to FillType. * Update test description. * Fix failing CI. * Fix DESO balance check in DAO coin limit order The way we were validating DESO balance made it so that we could only execute a "sell" of DESO if the resulting balance was above the sell amount. This made it so that the maximum amount of DESO we could sell in an order was ~(DESO balance / 2), rather than (DESO - fee). The fix was to move the execution of _connectBasicTransfer to *after* the point where all the validations were done. This had no impact other than fixing our validation logic. * Catch an error when parsing fill type * Add more uint8 overflow checks. Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * [stable] Release 2.1.2 (deso-protocol#344) * Hypersync (deso-protocol#346) * Add shared secret encryption/decryption * Make Encrypt Backwards Compatible * Fix comment * Minor fix * Skip unnecessary utxo deletes * Remove unnecessart balance nanos delete * Fix balance tests * Remove some unnecessary deletes * fix nft tests * Only flush PKID entries that are involved in swap identity transactions * Fix disconnect * Remove unnecessary profile deletes * Speed up balance entries and derived keys * Remove some bitcoin manager stuff * Profiles are slower but work * fix switch * try to fix test * [not safe] flush every 1k blocks * write our own encoders * Fix posts * Add hyper sync flag and fix comments * Fix... * More custom encoders * Add snapshot; Adjust DB; Add Hyper sync messages * Add snapshot to db_utils * Add snapshot everywhere; add encoders/decoders * Add checksum & ancestral records logic * Flushing ancestral records on utxo_view flush * Tests Fix * Remove delayed flush * Add utxokey encode/decode * Change dockerfile * Fix some missed fields * fix * fix * Fix appends in derivedKeyEntry * DeDupe * Fix tests * Debug snapshot; Add snapshot to all tests * Node sync fix * Initial snapshot peer messages * Snapshots are being sent * Sync works * Timing * Add ancestral records * First passing hyper sync with properly built & served ancestral records * Switch to Ristretto255 curve for checksum * DBEntry cleanup; parsing optimization * More cleanup; Change db prefixes; checksum updates * Curve games * Checksum games * Add worker pool checksum computation; checksum from block sync matches hyper sync; switch to deque in ancestral record memory * Add main and ancestral db semaphores * Fix merge conflicts * Fix checksum merge conflicts * Checksum optimization; add timer; add get snapshot validation * Another optimization * Correctly switch to block sync after state sync * Cleanup * 50% speed improvement; more cleanup * More cleanup * More comments * Document a lot * Txindex * Encoders everywhere; get rid of gob * Fix hot feed * Add encoder tests & fix postgres * Node testing framework initial * Add network bridges to node testing framework * Cleanup and document node testing; Add first tests * Add first hyper sync test * Clean formatting * Add restart tests; fmt * More tests * Some solid debugging * Debug sync past blocks after hyper sync * Test basic hyper sync followed by txindex * Txindex updates * Code review * Fix quit * wip tests passing * Slight refactor of nonexisting record thing * little wip tests passing * Fix bug around nested calls to PrepareAncestralRecordsFlush * Metadata and restarts * Rename snapshot.go:ProcessBlock to SnapshotProcessBlock * Murder the caterpillar * s/hyper-sync/hypersync * s/SnapshotProcessBlock/ProcessBlock where applicable (oops) * Add FIXMEs and TODOs for Piotr plus touchups * Murder the caterpillar (for real this time) Had a bug due to not converting ancestralDBEntry to NormalDBEntry * Checksum optimization, some cleanup * Smaller HashToCurve cache * Add blockheight to every encoder * Resurrect caterpillar (prev commit); rename encode/decode * Fix Fixmes * Bump the chunk size to 800MB * Re-murder caterpillar * Revert "Re-murder caterpillar" This reverts commit 1cdf31b. * Increase max messaage payload for hypersync * Revert "Re-murder caterpillar" This reverts commit 1cdf31b. * Updaqte memtablesize for badger * Update memtablesize * Add functionality needed to support Rosetta hypersync * Set badger memtable size to 4gb * Remove blockheight from encode/decode * Fix that dirty race condition in operationChannel; add block height to ancestral cache * Add performance * Fix shift bug * Update memtablesize for badger and fix hypersync restart issue * Add a lot of error handling; add status updates and initial snapshot recovery * Fix repost prefix overwrite * Remove broken snapshot * Config param rename * Fix random ordering of NFTBidEntry in UtxoOps * Stupido index * Fix NFT royalties encoder * Add snapshot block height node param * Fix flag * Add status db * Add log * glog global params * Add global params log * Pause miner and add some test logging for global params * Revert global params logging * Mine only when fully current * Add flags for timer * More concurrency control; prelude to snapshot recovery; log cooolors * Reorganize snapshot code * Fix green * Better logging * Better logging * Small adjustments * Fix * Another fix * Faster delete db * Add faster node erase * Logs and stop address relayer * Remove division bug * More seemless restart * Fix overlap, better logs * nit * Snapshot recovery mode * Fix logging; restart on invalid checksum * Encoder migration, add StatePrefixToDeSoEncoder map * Add encoder types and versions * Add migration to checksum computation * Proper add bytes in process chunk * Persist encoder checksums; add encoder block heights * Add migration on boot; switch migrations on snapshot process block * Fix disconnect checksum; test recovery mode; add migration tests * Fix block reward disconnect; simplify migrations; better recovery and restarts * Debug migrations; add ancestral delete * Fix nil pointer * Nice message when you close the node * Faster stops * Increase chunk size * Update restarts * Fix lib tests * Fix disconnects * Db update * Small cleanup * More comments * Fix pg db adapter * Archival mode * gofmt * More comments; automated encoder tests; fix txindex encoder * Cleanup; fix the NFT utxoop thing * Better chain state condition * Fetch blockNode by hash and fix blockHash equality * Resolve merge conflict agaaaain * Fix dao limit order fees * Fix nilptr reference; disaster condition in node; comments in snapshot * More cleanup and fix expected response * small comments * Make migration scheme utilize ForkHeight variables (reduce redundandcy) * Refactors; comments; fixes * Update comments * Fix * Add persistent flags * Comments, better version encoding * Run flags * The diamondhands holy pass is complete. * Add missing flag Co-authored-by: p <p> Co-authored-by: maebeam <maebeam0@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: root <root@ip-172-31-48-134.ec2.internal> * Fix postgres null balance entry. (deso-protocol#345) * Fix postgres null balance entry. * Check for nil in NewBalanceEntry. * Support DAOCoinLimitOrder cancels for derived keys (deso-protocol#347) * Fix regtest (deso-protocol#348) * Fix regtest * gofmt * Make --archival-mode true by default for now * Add a DB version to auto-change the datadir when major updates are made (deso-protocol#349) * [stable] Release 2.1.3 * Move float -> uint256 scaling calculation to a general purpose function (deso-protocol#350) * Move float -> uint256 scaling calculation to a general purpose function * add constants for NotGraylisted and NotBlacklisted (deso-protocol#196) * Diamondhands/fix hard fork issues (deso-protocol#351) * Fix nil TransactionSpendingLimitTracker issue * Fix nil check in logging of TransactionSpendingLimitTracker * Add nil checks for transaction spending limit maps Co-authored-by: lazynina <lazynina84@gmail.com> * Move Bitcoin RBF check to core * Refactor hypersync flags to be able to force blocksync or hypersync * Set mainnet block heings for DAODAO hard fork (deso-protocol#353) Hard fork date: Wednesday at 2pm PT (~8.5 days from now) Computing 1 week of blocks: * Actual block diff for 7 days from block explorer: - 128309 (today) * Mon May 09 2022 13:55:28 GMT-0700 (Pacific Daylight Time) - 126339 (7 days ago) * Mon May 02 2022 13:42:51 GMT-0700 (Pacific Daylight Time) - = 1970 blocks * From calculations: - 288 * 7 - 2016 * Sanity-check: - (2016-1970) = 46 blocks diff - ~2.3% difference = 3.8 hours (not a big deal) - This is actually good because it means that we have some buffer Computing blocks from now until hard fork * 128309 - Mon May 09 2022 13:55:28 GMT-0700 (Pacific Daylight Time) * Add precisely 9 days of blocks: - 9 * 288 = 2592 blocks - 128309 + 2592 = 130901 * [stable] Release 2.2.0 * [stable] Release 2.2.1 Bumping release to fix a CI issue * Allow SignTransactionBytes to sign derived or non-derived key * Remove txindex reliance on snapshotting * Expose postgres db to other services * Fix bug in limit order logic around small orders We discovered a minor bug whereby orders could be submitted that are so small that they can be filled without the other side of the trade paying anything. While this is generally harmless, some error-checking logic made it so that new orders would be rejected if the order on the top of the book had this issue. This change resolves this issue and adds thorough testing around it. Added test case to reproduce bug Fix bug and add comprehensive test cases Fix test issue * [stable] Release 2.2.2 * replace sqs code & pull in changes * formatting & fix tests * enhance sqs logging * enhance sqs logging * enhance sqs logging * enhance sqs logging * enhance sqs logging Co-authored-by: maebeam <maebeam0@gmail.com> Co-authored-by: Erwin Willems <github@reflexlight.nl> Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: Piotr Nojszewski <piotrek.nojszewski@gmail.com> Co-authored-by: Hunter Paulson <hpaulson@seismiccore.com> Co-authored-by: Camden Foucht <Foucht@gmail.com> Co-authored-by: thoughtlesslabs <john@thoughtlesslabs.com> Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: Matt Pitts <mattapitts@gmail.com> Co-authored-by: diamondhands0 <81935176+diamondhands0@users.noreply.github.com> Co-authored-by: kanshi <andrzej@yexperiment.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com> Co-authored-by: BitClout.com/u/tijn <69529928+tijno@users.noreply.github.com> Co-authored-by: superzordon <88362450+superzordon@users.noreply.github.com> Co-authored-by: Juan Carlos del Valle <jc.ekinox@gmail.com> Co-authored-by: elmasapp <96642041+elmasapp@users.noreply.github.com> Co-authored-by: Don Hardman <59067742+donhardman@users.noreply.github.com> Co-authored-by: redpartyhat <81658108+redpartyhat@users.noreply.github.com> Co-authored-by: Michel Majdalani <43356588+MichelMajdalani@users.noreply.github.com> Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> Co-authored-by: iamsofonias <99746187+iamsofonias@users.noreply.github.com> Co-authored-by: lazynina <lazynina84@gmail.com> Co-authored-by: root <root@ip-172-31-48-134.ec2.internal>
piratekev
added a commit
to TheBitgram/core
that referenced
this pull request
Jun 13, 2022
* Use nil blockhashex on mempool transactions (deso-protocol#140) * Add a list of nodes maintained by the core developer team (deso-protocol#145) Co-authored-by: Erwin Willems <github@reflexlight.nl> Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> * Limit mempool txn size to half of block size * Fix BTC buy check * Increase logging on BlockCypherCheckBitcoinDoubleSpend * Simpler derived keys (deso-protocol#148) * Simplify * Fix Co-authored-by: p <p> * Added additional nodes (deso-protocol#149) * add new node (deso-protocol#152) * Missing comma on newest node (deso-protocol#157) * Add DeversoNode (deso-protocol#156) * add double spend detection source in error message (deso-protocol#159) * add double spend detection source in error message * Add txn hash to double-spend error logging Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Add support for nft transfer notifications (deso-protocol#158) * Add support for nft transfer notifications * add trailing comma in nodes.go * Move vendored libraries to separate repos (deso-protocol#160) * [stable] Release 1.2.8 * update GetPostsPaginatedForPublicKeyOrderedByTimestamp to take in nftsRequired boolean (deso-protocol#161) * Adding node (deso-protocol#163) * Split block view and tests (deso-protocol#164) * expose the prefix for the index from public key to deso balance (deso-protocol#168) * expose the prefix for the index from public key to deso balance * expose prefix for creator deso locked * Factor the double-spend check into its own function (deso-protocol#165) * Factor the double-spend check into its own function * return boolean indicating if error is from double spend detection or not (deso-protocol#167) * return boolean indicating if error is from double spend detection or not * flip return values order Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Ln/fix minor disconnect bid issue (deso-protocol#169) * [stable] Release 1.2.9 * Buy Now NFTs with Buy Now Price (FORKING CHANGE) (deso-protocol#166) * Refactor connectAcceptNFTBid to move logic that handles transferring the NFT, computing royalties and payouts, and executing payouts so it can be reused in connectNFTBid to support Buy now NFTs * update disconnect logic for nft bid to account for buy now NFTs * update tests and disconnect logic for NFT bids * finish buy now test and add more comments around test * Add NFTBidCreatorPublicKey, NFTBidBidderPublicKey, NFTBidCreatorRoyaltyNanos to UtxoOperation for NFTBid transactions * remove unused BidderInputs from NFTBidMetadata, add comments above IsBuyNowKey * update notifications for NFT bid on buy now nft * remove commented out line * clean up comments * add block height to trigger Buy Now NFTs * fix tests involving buy now nfts * address RPH feedback * address a lot of DH feedback * Update to have a separate price for Buy Now NFTs instead of using MinBidAmountNanos * force Buy Now price to be greater than min bid amount * check in current progress on tests * finish out testing of Buy now NFT with separate buy now price * submitting a bid on serial number 0 will not trigger a buy now operation. You must specifically bid on a serial number for a buy now to occur * resolve merge conflicts with main * resolve conflicts in block_view_nft.go * set rosetta fields appropriate in helpConnectNFTSold * remove rosetta fields from initial construction * diamondhands review, clean up, fix a couple minor bugs * Moved the _helpConnectNFTSold block to closer to where it was previously to make reviewing easier. * Caught an error in CreateAcceptNFTBid that wasn't being caught before. * Fixed the ordering of adding to the utxoOperations list in _helpConnectNFTSold. The ops were being added in the wrong order before. In particular, we were doing: - _connectBasicTransfer - Spending the BidderInputs - Adding the royalties But the UtxoOps were `[bidder inputs, add royalties, connect basic transfer]`, when they should have been `[connect basic transfer, bidder inputs, add royalties]`, i.e. same order as the operations we performed. * Fixed some comments here or there Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * DAO coins (deso-protocol#174) * Refactor connectAcceptNFTBid to move logic that handles transferring the NFT, computing royalties and payouts, and executing payouts so it can be reused in connectNFTBid to support Buy now NFTs * update disconnect logic for nft bid to account for buy now NFTs * update tests and disconnect logic for NFT bids * finish buy now test and add more comments around test * Add NFTBidCreatorPublicKey, NFTBidBidderPublicKey, NFTBidCreatorRoyaltyNanos to UtxoOperation for NFTBid transactions * remove unused BidderInputs from NFTBidMetadata, add comments above IsBuyNowKey * update notifications for NFT bid on buy now nft * remove commented out line * clean up comments * add block height to trigger Buy Now NFTs * fix tests involving buy now nfts * address RPH feedback * address a lot of DH feedback * Update to have a separate price for Buy Now NFTs instead of using MinBidAmountNanos * force Buy Now price to be greater than min bid amount * check in current progress on tests * finish out testing of Buy now NFT with separate buy now price * submitting a bid on serial number 0 will not trigger a buy now operation. You must specifically bid on a serial number for a buy now to occur * resolve merge conflicts with main * resolve conflicts in block_view_nft.go * set rosetta fields appropriate in helpConnectNFTSold * remove rosetta fields from initial construction * Initial check-in of DAO Coin PR * add migration file * add more comments and make some code a little cleaner * address piotr feedback * address more Piotr feedback and add network tests * update HasPurchased in GetHoldings and GetHolders * fix indent on OperationTypeDAOCoinTransfer * add transfer restrictions to DAO coins * update postgress accordingly{ * update postgres handling * fix rule error capitalization * diamondhands review, clean up, fix a couple minor bugs * Moved the _helpConnectNFTSold block to closer to where it was previously to make reviewing easier. * Caught an error in CreateAcceptNFTBid that wasn't being caught before. * Fixed the ordering of adding to the utxoOperations list in _helpConnectNFTSold. The ops were being added in the wrong order before. In particular, we were doing: - _connectBasicTransfer - Spending the BidderInputs - Adding the royalties But the UtxoOps were `[bidder inputs, add royalties, connect basic transfer]`, when they should have been `[connect basic transfer, bidder inputs, add royalties]`, i.e. same order as the operations we performed. * Fixed some comments here or there * Fix merge conflict with test * Merbe _connectCreatorCoinTransfer and _connectDAOCoinTransfer * [REQUIRES RESYNC] Cleanup and rename CoinEntry -> CreatorCoinEntry * Move HelpConnectCoinTransfer * Add missing import * [REQUIRES RESYNC] Switch to uint256 for coin balances * In txns: - DAOCoinMetadata * CoinsToMintNanos * CoinsToBurnNanos - DAOCoinTransferMetadata * DAOCoinToTransferNanos In db structs: - CoinEntry * CoinsInCirculation * Didn't need CoinWatermarkNanos because CreatorCoins don't go over uint64 - BalanceEntry * BalanceNanos All tests passing * Add overflow tests for DAO coins * Nit: Add error logging for balance lookup Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Ln/nft splits (deso-protocol#173) * Refactor connectAcceptNFTBid to move logic that handles transferring the NFT, computing royalties and payouts, and executing payouts so it can be reused in connectNFTBid to support Buy now NFTs * update disconnect logic for nft bid to account for buy now NFTs * update tests and disconnect logic for NFT bids * finish buy now test and add more comments around test * Add NFTBidCreatorPublicKey, NFTBidBidderPublicKey, NFTBidCreatorRoyaltyNanos to UtxoOperation for NFTBid transactions * remove unused BidderInputs from NFTBidMetadata, add comments above IsBuyNowKey * update notifications for NFT bid on buy now nft * remove commented out line * clean up comments * add block height to trigger Buy Now NFTs * fix tests involving buy now nfts * address RPH feedback * address a lot of DH feedback * Update to have a separate price for Buy Now NFTs instead of using MinBidAmountNanos * force Buy Now price to be greater than min bid amount * check in current progress on tests * finish out testing of Buy now NFT with separate buy now price * submitting a bid on serial number 0 will not trigger a buy now operation. You must specifically bid on a serial number for a buy now to occur * resolve merge conflicts with main * resolve conflicts in block_view_nft.go * check in current progress on NFT splits * start test of nft splits * add IsDeleted function to pkid entries * add more error case testing * add comments, resolve TODOs, and check for overflow * set rosetta fields appropriate in helpConnectNFTSold * remove rosetta fields from initial construction * refactor construction of additional royalties to reduce code duplication * go fmt project * Review splits. Ready to ship! * Made custom deterministic serializers rather than using gob * Changed interface to use public key, hiding PKID * Added SwapIdentity to the test on splits. Looks good * Fixed a minor overflow bug * TODO: Add a test for overflows * add more checks for overflow and add more testing for overflow * add one more test on overflow for good measure Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Fix bug introduced by NFT splits (deso-protocol#176) * Add Tunel node (deso-protocol#177) * Check CreatorPKID instead of HODLerPKID when iterating over utxoview (deso-protocol#178) * Move blockheights into DeSoParams and set values accordingly (deso-protocol#171) * Move blockheights into DeSoParams and set values accordingly * fix test * remove unnecessary update to set blockheight * resolve conflicts with main * fix alignment and remove original constants for block heights * go fmt constants.go * fix BuyNowAndNFTSplitsBlockHeight usage in extractAdditionalRoyaltyMap * use params.DAOCoinBlockHeight * use params.BuyNowAndNFTSplitsBlockHeight * Embed fork heights in a struct * Make testnet heights equal to previous values and fix broken test Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Messages V3 (deso-protocol#182) * Finish connect/disconnect logic * Add requests * Add Party Entry, Start Connect/Disconnect * Finish connecting party messages; all core logic done * Small rename * Fix messaging key inconsistencies; general clean * Cleanup code; Add tests for messaging keys * Fix weird indents * Add initial tests for message parties * Finish testing * Thoroughly tested and some more cleanup * Add support for get/send backend endpoints * Update backend endpoint * Add block height * Lang * Finish core & add tests * Finish testing messaging keys; hard part done, now onto private message tests * Cleanup test comments * Fix private message connect/disconnect; cleanup * Add more tests * Fix setters/getters for message entries * All done & tested * Final cleanup * Add network tests * Fix isRecipient * Simplify messaging key entry * Simplify messaging key entry (deso-protocol#181) * Unencrypted groups support * Fork heights * Clean version code * Nits * Upgrade some comments * Clean up messages v3 - Fixed isDeleted bug in the fetching of group messages - Rename everything to be group-oriented: * MessagingKey -> MessagingGroupKey * MessagingKeyEntry -> MessagingGroupEntry * recipient -> member * MessagingRecipient -> MessagingGroupMember - Refactor some db stuff. Mainly added GroupOwnerPublicKey to the MessagingGroupMember, and cleaned up the back-reference there. * Test fix * Final passerinio * Good luck pass; and prefix fix * Unfix prefix Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Fix instances where a pointer is taken of a map key iterator (deso-protocol#183) * Fix another iterator pointer issue (deso-protocol#184) * Fix stubborn NFT splits bug due to reordering of royalties in map (deso-protocol#188) * [stable] Release 2.0.0 (deso-protocol#190) * Set block height triggers for DAO Coins, NFTs v2, and Messages v3 * [stable] Release 2.0.1 * Fix sql statements for DAOCoin/Nft (deso-protocol#193) * Add new constant to constants file to represent the NodeSource map key in Extra Data (deso-protocol#191) * Adding entre's node. (deso-protocol#179) * nft and dao coin notifications after fork (deso-protocol#185) * add support for DAO coin and nft royalty notifications * updates to better support notifications on the frontend * fix issue with pointers in for loops * merge in main and copy iterator * fix creator pub key * Fix creator public key in nft bid * skip affected pub key if royalty amount is 0 * simplify owner public key in NFT bid case * simplify creator public key in Accept NFT Bid case * add deso royalty map to utxo * make the NFT bid case mirror accept NFT bid * [stable] Release 2.0.2 * Update Postgres handling of DAO Coins and NFTs (deso-protocol#194) * [stable] Release 2.0.3 * Add a SQL adapter for BlockHash that makes it work with bun (deso-protocol#204) This will allow us to upgrade to Postgres v14 bu using the bun library rather than the super old go-pg library, which is still stuck at v10 * fix DAO coin postgres issues (deso-protocol#208) * [stable] Release 2.0.4 (deso-protocol#209) * create maps for additional royalties when converting to/from PGPostEntry before copying values into it (deso-protocol#212) * create maps for additional royalties when converting to/from PGPostEntry before copying values into it * Add debugging to error path * Update encoding of pkid in AdditionalNFTRoyaltiesToCoinsBasisPoints * Add group messaging support * Add messaging group tables * Fix small bug in down of migration Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Return error if error during txindex db update (deso-protocol#220) * Fix Issue with ParentStakeID set when modifying Post (deso-protocol#222) * move ComputeMetadata to a switch-case and glog errors for Like and SubmitPost Txns * don't return an error from ComputeTransactionMetadata, just glog it * Create accept bid for buynow when create bid in postgres (deso-protocol#226) * Create accept bid for buynow when create bid in postgres * Update lib/postgres.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * add AcceptedBlockHeight to NFTBidEntry (REQUIRES RESYNC) (deso-protocol#224) * add AcceptedBlockHeight to NFTBidEntry * Set Accepted on PGNFTBid is AcceptedBlockHeight is not nil * only copy AcceptedBlockHeight if not nil * add txindex metadata for Burns and Accept NFT Transfers (deso-protocol#228) * Update nodes.go (deso-protocol#202) * Update nodes.go * Update nodes.go * Add upcoming overclout node (deso-protocol#219) * Add upcoming overclout node * Update lib/nodes.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Fix get-all-messaging-keys & add base point test (deso-protocol#217) * add remaining UtxoTypes to UtxoType.String() (deso-protocol#229) * [stable] Release 2.0.6 (deso-protocol#231) * ch pgBidNft is not nil (deso-protocol#232) * [stable] Release 2.0.7 (deso-protocol#233) * Ln/spending limits (deso-protocol#216) * Fix profile vulnerability. (#1) * lower height * Add transaction spending limit for derived keys save current progress on derived key spending limit add more testing, remove usage of pointers in limit map keys * separate SET and TRACK block heights for derived key transaction spending limits * remove weird test * address Piotr feedback - encoders/decoders for map keys, add comments, refactor inline functions * go fmt project * add TransactionSpendingLimitTracker and Memo to PGDerivedKey * updates to converting DAO/NFT/CC operations to string for backend usage * compare operation limit keys by hex encoding * add more tests * diamondhands edits * merge in main * Add ZeroBlockHash and use ZeroBlockHash and ZeroPKID in map keys * diamondhands touchups * set regtest forkheights to 0 * add test that derived key can sign its own transactions and spend its own money * Address fixmes * rebuild * add test case for minting NFT * reduce nested if statements in block_view.go * Update lib/block_view_types.go Co-authored-by: redpartyhat <81658108+redpartyhat@users.noreply.github.com> Co-authored-by: maebeam <maebeam0@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Extra Data on Posts, NFTs, Messages, Messaging Groups, and Derived Keys (deso-protocol#213) * Add ProfileExtraData to ProfileEntry struct * add test to ensure extra data comes through. update calls to CreateUpdateProfileTxn, add postgres support * Update lib/block_view_profile_test.go * Review and cleanup * Add one more fixme * update profile test for extra data * ExtraData on NFT / Message / GroupMessagingKey / DerivedKey entries (deso-protocol#221) * Return error if error during txindex db update (deso-protocol#220) * Add extra data to NFT, Message, GroupMessagingKey, and DerivedKey entries Return error if error during txindex db update (deso-protocol#220) add tests for adding extra data on NFT, private message, messaging group, and derived key entries add comment on extra data * Fix Issue with ParentStakeID set when modifying Post (deso-protocol#222) * move ComputeMetadata to a switch-case and glog errors for Like and SubmitPost Txns * don't return an error from ComputeTransactionMetadata, just glog it * Create accept bid for buynow when create bid in postgres (deso-protocol#226) * Create accept bid for buynow when create bid in postgres * Update lib/postgres.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * add AcceptedBlockHeight to NFTBidEntry (REQUIRES RESYNC) (deso-protocol#224) * add AcceptedBlockHeight to NFTBidEntry * Set Accepted on PGNFTBid is AcceptedBlockHeight is not nil * only copy AcceptedBlockHeight if not nil * add txindex metadata for Burns and Accept NFT Transfers (deso-protocol#228) * Update nodes.go (deso-protocol#202) * Update nodes.go * Update nodes.go * Add upcoming overclout node (deso-protocol#219) * Add upcoming overclout node * Update lib/nodes.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Fix get-all-messaging-keys & add base point test (deso-protocol#217) * add remaining UtxoTypes to UtxoType.String() (deso-protocol#229) * diamondhands touchups with some FIXMEs for @lazynina * delete fixmes about deleting protected keys * add test for encoding/decoding message entries for backwards compatibility * add test for encoding/decoding messaging group entries for backwards compatibility Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> Co-authored-by: elmasapp <96642041+elmasapp@users.noreply.github.com> Co-authored-by: Don Hardman <59067742+donhardman@users.noreply.github.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Update lib/blockchain.go Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> Co-authored-by: elmasapp <96642041+elmasapp@users.noreply.github.com> Co-authored-by: Don Hardman <59067742+donhardman@users.noreply.github.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com> * Gofmt the whole repo * simplify extra data handling in CreateAuthorizeDerivedKey (deso-protocol#235) * operationtype typo in disconnect (deso-protocol#238) * Fix postgres down migrations. (deso-protocol#276) * Make derived key txn construction accept a spending limit hex, not object (deso-protocol#297) * Make derived key txn construction accept a spending limit hex, not object * Update lib/block_view_derived_key.go * add exists declaration back. Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Use NewPKID to get bidder PKID if bidder's profile is not found (deso-protocol#262) * Feature/add dao coin limit order transaction (deso-protocol#307) * Add DAO Coin Limit Order network skeleton. (deso-protocol#239) * Add DAO Coin Limit Order constants. (deso-protocol#240) * Add DAO Coin Limit Order mempool skeleton. (deso-protocol#243) * Add DAO Coin Limit Order mempool skeleton. * Restore removed imports. * Side-step unused var error. * Add DAO Coin Limit Order blockchain skeleton. (deso-protocol#242) * Add DAO Coin Limit Order blockchain skeleton. * Restore removed imports. * Add DAO Coin Limit Order block view skeleton. (deso-protocol#241) * Add DAO Coin Limit Order block view skeleton. * Restore removed import. * Side-step unused var error. * Add DAO Coin Limit Order block view flush skeleton. (deso-protocol#244) * Add DAO coin limit order db prefixes. (deso-protocol#245) * Mf/add dao coin limit order key builders (deso-protocol#246) * Add DAO coin limit order key builders. * Use uint32s for enums. * Remove unnecessary line. * Mf/add dao coin limit order db utils (deso-protocol#247) * Add DAO coin limit order db utils. * Add FromBytes skeleton method. * Fix FromBytes operation happens in-place. * Mf/add dao coin limit order postgres skeleton (deso-protocol#248) * Add DAO coin limit order postgres skeleton. * Fix spacing/tabs. * Fix method call capitalization. * Convert uint256 types to strings. * Add DAO coin limit order network metadata. (deso-protocol#249) * Revert DAO coin limit order enums to uint8s. (deso-protocol#250) * Add DAO coin limit order connect transaction logic. (deso-protocol#259) * Mf/add dao coin limit order txn testing (deso-protocol#263) * Switch DAO coin order limit quantity to pointer. (deso-protocol#265) * Add more DAO coin limit order testing scenarios. (deso-protocol#267) * Convert DAO coin limit order price from BigFloat to Uint256. (deso-protocol#266) * Convert DAO coin price to uint256. * Complete conversion of DAO limit order price to uint256. * Use Lt, Gt instead of Cmp. * Start inverting price comparison logic. * add function to calculate total cost of an order (deso-protocol#268) * add function to calculate total cost of an order * Remove Conditional from DBGetMatchingDAOCoinBidOrders that didn't apply, Fix IsBetterAskThan and IsBetterBidThan (deso-protocol#269) * add check that order is better than last seen order when getting next orders to fill, fix txn metadata for m1's txn in test * use total cost calculating function to accurately get total cost of orders in blockchain.go, set prefixKey correctly for DBGetMatchingDAOCoinAskOrders Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Mf/track dao coin limit order utxos (deso-protocol#270) * Track DAO coin limit order UTXOs. * Add disconnect logic. * Fix failing test. * Update lib/block_view_dao_coin_limit_order.go * Add DAO coin limit order postgres structs. (deso-protocol#271) * Add DAO coin limit order postgres structs. * Update dao coin limit order postgres migration. * Fill out LeftPaddedHexToUint256. * Add DAO coin limit order postgres tests. (deso-protocol#273) * Remove logging before flags error. (deso-protocol#274) * Fix DAO coin down migration. (deso-protocol#275) * Fix some postgres testing issues. (deso-protocol#277) * Add additional DAO coin limit order test and resolve bugs it surfaces. (deso-protocol#278) * Add additional DAO coin limit order testing. * Refactor block view to use db adapter for DAO coin limit orders. * Set up multi-order test. * Give m0 more initial in testing. * Fix failing txn spend amt test. * Fix updating order's quantity bug. * Add helper function to query for all orders. * Fix by transactor PKID prefix bug. * Mf/better dao coin limit order var naming (deso-protocol#279) * Refactor DAO coin limit order variable naming. * Rename order to matchingOrder. * Test DAO coin limit order total cost doesnt overflow validation. (deso-protocol#280) * Mf/allow dao coin limit order cancellations (deso-protocol#281) * Allow DAO coin limit order cancellations. * Fix failing postgres test. * Add more comments. * Add failing test to show no transactorPKID in index bug. * Update helper utils to new order schema. * Checkpoint. Store progress updating connect logic. * Another checkpoint. Stash progress in connect logic. * Commit checkpoint. Updated all of block view. * Start rebuilding tests. * Checkpoint. Continue updating tests. * Add back endpoint to query open orders by transactor PKID. * Add function to get all limit orders for a DAO coin pair. * Clean up postgres. * Convert price from uint256 to big.Float. * Add uint256 ScaledPrice field. * Continue turning back-on more tests. * Fix more tests. * Checkpoint. Continue fighting tests. * Continue fixing tests. * Fix last two tests. * Revised a lot of things, but mainly _connect logic. Let's have a meeting to discuss. * Convert PKID vars to PublicKeys. * Start fixing tests. * Fix tests by removing reflect.DeepEqual. * Switch buying and selling DAO coins back to PKIDs. * Fix a few more failing tests. * Must compare transactor buying qty to matching selling qty. * Continue fixing tests. * Get cancellation tests passing. * Only update orders within the same block height. * Fix small postgres bug. * Ln/dao coin limit order txindex (deso-protocol#283) * initial check in for txindex for dao coin limit orders * Use new struct to track fulfilled orders * update naming convention * Mf/add safe math uint256 utils (deso-protocol#285) * Add SafeMathUint256 utils. * Convert some DAO coin limit order math to safe math. * Save me some whitespace. * Mf/add safe math uint64 utils (deso-protocol#287) * Add SafeMathUint256 utils. * Convert some DAO coin limit order math to safe math. * Save me some whitespace. * Add SafeMathUint64 utils. * Update DAOCoinLimitOrderMetadata type so it can be serialized to json (deso-protocol#286) * Update DAOCoinLimitOrderMetadata type so it can be serialized to json * Re-autoformat comments * Address feedback on sorting slices * Address feedback in connect transaction function * Mf/add dao coin transfer restriction validations (deso-protocol#288) * Add DAO coin limit order transfer restriction validations. * Make a bit cleaner. * Keep full error message. * Add UTXO view API DAO coin limit order getters. (deso-protocol#290) * Add UTXO view API DAO coin limit order getters. * Make merging database and UTXO values more consistent. * Only compute order map key once. * Ln/dao coin limit order disconnect (deso-protocol#289) * Add temporary postgres DAO coin limit order solution. (deso-protocol#292) * Initial check in for spending limits on DAO Coin Limit Orders (deso-protocol#293) * Initial check in for spending limits on DAO Coin Limit Orders * add comments, rename key to DAOCoinLimitOrderLimitKey, initialize map in copy, add tests for spending limit on dao coin limit orders * Add additional DAO coin limit order tests. (deso-protocol#291) * Ln/dao coin limit order cancellation validations (deso-protocol#294) * Add additional DAO coin limit order tests. * fix infinite loop with last seen order, fix validations around cancellations, update tests Co-authored-by: mattfoley8 <isaac@deso.org> * Address fixmes: add sanity checks and comments. Add isDeleted checks and new RuleErrors (deso-protocol#295) * Ln/dao coin limit order loop till filled (deso-protocol#296) * Loop until order is filled, fix db utils for getting next order, add more tests, fix copy DAOCoinLimitOrderEntry to not return error * leave comment in test for diamondhands * add balance checks in latest test * Diamondhands touch-ups * Add postgres CI whitelisted testing. (deso-protocol#298) * Add postgres CI whitelisted testing. * Run migrations before testing. * Map postgres port to localhost. * Empty commit to test caching dependencies. * Test migration rollbacks as well. * Add newline at EOF. * Mf/add additional dao coin order book tests (deso-protocol#299) * Add additional DAO coin order book tests. * Dont say bid/ask. * Add go-fmt to CI. (deso-protocol#300) * Add go-fmt to CI. * Fix failing gofmt error. * Address DH fixmes (deso-protocol#301) * Add another test. (deso-protocol#302) * Fix inputs from transactor to account for fee AND DESO spent purchasing DAO Coins, fix postgres, and add FeeNanos to txn metadata (deso-protocol#303) * Fix inputs from transactor to account for fee AND DESO spent purchasing DAO Coins, fix postgres, and add FeeNanos to txn metadata * Clone quantity to be safe * go fmt blockchain.go * always write utxo ops to badger (deso-protocol#304) * always write utxo ops to badger * run disconnect for pg * fix table name for dao coin limit order metadata * Add local postgres testing setup. (deso-protocol#305) * Make derived key txn construction accept a spending limit hex, not object (deso-protocol#297) (deso-protocol#306) * Make derived key txn construction accept a spending limit hex, not object * Update lib/block_view_derived_key.go * add exists declaration back. Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: diamondhands0 <81935176+diamondhands0@users.noreply.github.com> * Add fee nanos to the to bytes and from bytes logic for DAO Coin Limit Orders (deso-protocol#308) * Error if matching own order. (deso-protocol#309) * Make API getter functions public (deso-protocol#310) * Mf/add operation type column (deso-protocol#311) * Rename quantity to buy -> quantity to fill. * Checkpoint passing tests. * Get postgres tests working. * Add helper function and tests. * Continue adding tests for operation type quantity math. * Complete BID-BID calculation logic. * Build out ask-ask case. * Use named return values instead of a struct. * Allow for param specifying different transactor quantity to fill. * Implement logic in db_utils. * Refactor calculate test into own test. * Update blockchain.go w/ new quantity calculation. * Throw error if updating order w/ different operation type. * Update connection logic. * Fix failing derived key test. * Document off-by-one bug. * Add check on FeeNanos in DAO Coin Limit Order connect logic * Check for overflow before mutliplying fee nanos * revert changes to UpdateGlobalParams txn construction * Update w/ PR review feedback. * Make price calculators var naming a bit more consistent. * Remove DAO prefix from filled order struct qty fields. * Empty commit to trigger CI. * Fix off-by-one bug. * Fix floating point issues in tests, and implement fixed-point conversion scheme * Some touch-ups to the order matching logic. * Add a FIXME for an ultimate sanity-check in _connectDAOCoinLimitOrder * Add money printer prevention validation. * WIP - fix prevBalances, but sanity-check fails * Fix prevBalances bug and fix sanity-check after * Resolve merge conflicts. * Fix tests with a workaround on fee computation * diamondhands holy tidy pass * Fix failing postgres test. Co-authored-by: lazynina <lazynina84@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Add DAO coin limit order blockheight validation. (deso-protocol#314) * Add DAO coin limit order blockheight validation. * Empty commit to retrigger CI. * Set DAO coin limit order block height in derived key test. * Fix get next orders to fill qty check. (deso-protocol#315) * Add some cosmetic updates. (deso-protocol#316) * don't return error from functions that don't generate errors (deso-protocol#318) * TotalInput and TotalOutput should consider inputs and outputs to/from bidders and difference should equal fee (deso-protocol#317) * TotalInput and TotalOutput should consider inputs and outputs to/from bidders and difference should equal fee * Update lib/block_view_dao_coin_limit_order.go Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> * Update lib/block_view_dao_coin_limit_order.go Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> * Test submitting ASK order and matching existing BID. (deso-protocol#319) Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: iamsofonias <99746187+iamsofonias@users.noreply.github.com> Co-authored-by: diamondhands0 <81935176+diamondhands0@users.noreply.github.com> Co-authored-by: lazynina <lazynina84@gmail.com> * Mf/additional dao coin limit order testing (deso-protocol#324) * Only copy TransactionSpendingLimitTracker if non-nil, (tsl *TransactionSpendingLimit) Copy() should return nil if tsl is nil (deso-protocol#325) * Mf/add dao coin limit order id field (deso-protocol#327) * Start adding DAO coin limit order ID field. * Add OrderID field to connect logic cancellation. * Simplify MapKey now that we have an order uuid. * Fix failing postgres tests. * Fix failing derived key test. * Fix comments / spacing. * Add OrderID to FilledOrder. * Store copy of order in prev in case of rollback. * Add additional postgres indices. * Improve parity between Postgres and Badger result set ordering. * Address review feedback pt 1. * Only store metadata CancelOrderID in bytes if present. * Use maxHash as max block hash. * Mf/dao coin order metadata optional fields (deso-protocol#329) * DAO coin order metadata optional fields. * Encode optional fields with byte count prefix. * Clean up blockchain.go branching logic. * Clean up more branching logic. * Better English in comments. * Remove TransactorPKID from primary DAO coin order book index. (deso-protocol#331) * Remove TransactorPKID from primary DAO coin order book index. * Fix failing postgres test. * Fix intermittent badger CI failure. * Remove sorting by quantity. * Remove price and blockheight from by transactor index. * Mf/test swap identity and dao coin orders (deso-protocol#330) * Test swap identity and DAO coin orders. * Test moar swapping identity + order book cases. * Make postgres tests pass. * Update m0s PKID in tests. * Add test that owner can deauthorize derived key with new transaction spending limit (deso-protocol#326) * [TESTNET HARD FORK]: Update TESTNET block heights for DAOSwap hard fork (deso-protocol#332) * [TESTNET HARD FORK]: Update TESTNET block heights for DAOSwap hard fork The following hard forks are triggering on Testnet on Wed Apr 20 @ 9am ET: ExtraDataOnEntriesBlockHeight: uint32(304087), DerivedKeySetSpendingLimitsBlockHeight: uint32(304087), DerivedKeyTrackSpendingLimitsBlockHeight: uint32(304087), DAOCoinLimitOrderBlockHeight: uint32(304087), * Update DerivedKeyTrackSpendingLimitsBlockHeight * [stable] Release 2.1.0 (deso-protocol#333) * Mf/add unused bidder inputs test (deso-protocol#334) * [stable] Release 2.1.1 (deso-protocol#335) * Mf/additional bidder input tests (deso-protocol#339) * Add tests for metadata FeeNanos. (deso-protocol#337) * Add tests for metadata FeeNanos. * Revert change to connect logic. * Mf/bidder input edge case testing (deso-protocol#338) * Bidder input edge case testing. * Add DAO<-->DAO coin test. * Fix test for including spent UTXOs. * Try increasing fee rate and retesting. * Unused UTXOs get refunded. * Use m2's public key. * Mf/add limit order metadata order type field (deso-protocol#342) * Add limit order metadata OrderType field. * Update postgres moar. * Dynamically determine order book transactor fees. * Add helper function for calculating FeeNanos. * Use uint8s for enums. * Add txn metadata validations. * Small refactor as vars used only once. * Add ImmediateOrCancel order type. * Allow zero price on IOC and FOK orders. * Allow user to specify exchange rate on IOC and FOK orders. * Add tests for market ASK orders. * Store OrderEntry.OrderType in the database. * diamondhands touchups * Require OrderType must be specified. No default value. * Refactor IsMarketOrder util function location. * Validate txn metadata. * Rename OrderType to FillType. * Update test description. * Fix failing CI. * Fix DESO balance check in DAO coin limit order The way we were validating DESO balance made it so that we could only execute a "sell" of DESO if the resulting balance was above the sell amount. This made it so that the maximum amount of DESO we could sell in an order was ~(DESO balance / 2), rather than (DESO - fee). The fix was to move the execution of _connectBasicTransfer to *after* the point where all the validations were done. This had no impact other than fixing our validation logic. * Catch an error when parsing fill type * Add more uint8 overflow checks. Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * [stable] Release 2.1.2 (deso-protocol#344) * Hypersync (deso-protocol#346) * Add shared secret encryption/decryption * Make Encrypt Backwards Compatible * Fix comment * Minor fix * Skip unnecessary utxo deletes * Remove unnecessart balance nanos delete * Fix balance tests * Remove some unnecessary deletes * fix nft tests * Only flush PKID entries that are involved in swap identity transactions * Fix disconnect * Remove unnecessary profile deletes * Speed up balance entries and derived keys * Remove some bitcoin manager stuff * Profiles are slower but work * fix switch * try to fix test * [not safe] flush every 1k blocks * write our own encoders * Fix posts * Add hyper sync flag and fix comments * Fix... * More custom encoders * Add snapshot; Adjust DB; Add Hyper sync messages * Add snapshot to db_utils * Add snapshot everywhere; add encoders/decoders * Add checksum & ancestral records logic * Flushing ancestral records on utxo_view flush * Tests Fix * Remove delayed flush * Add utxokey encode/decode * Change dockerfile * Fix some missed fields * fix * fix * Fix appends in derivedKeyEntry * DeDupe * Fix tests * Debug snapshot; Add snapshot to all tests * Node sync fix * Initial snapshot peer messages * Snapshots are being sent * Sync works * Timing * Add ancestral records * First passing hyper sync with properly built & served ancestral records * Switch to Ristretto255 curve for checksum * DBEntry cleanup; parsing optimization * More cleanup; Change db prefixes; checksum updates * Curve games * Checksum games * Add worker pool checksum computation; checksum from block sync matches hyper sync; switch to deque in ancestral record memory * Add main and ancestral db semaphores * Fix merge conflicts * Fix checksum merge conflicts * Checksum optimization; add timer; add get snapshot validation * Another optimization * Correctly switch to block sync after state sync * Cleanup * 50% speed improvement; more cleanup * More cleanup * More comments * Document a lot * Txindex * Encoders everywhere; get rid of gob * Fix hot feed * Add encoder tests & fix postgres * Node testing framework initial * Add network bridges to node testing framework * Cleanup and document node testing; Add first tests * Add first hyper sync test * Clean formatting * Add restart tests; fmt * More tests * Some solid debugging * Debug sync past blocks after hyper sync * Test basic hyper sync followed by txindex * Txindex updates * Code review * Fix quit * wip tests passing * Slight refactor of nonexisting record thing * little wip tests passing * Fix bug around nested calls to PrepareAncestralRecordsFlush * Metadata and restarts * Rename snapshot.go:ProcessBlock to SnapshotProcessBlock * Murder the caterpillar * s/hyper-sync/hypersync * s/SnapshotProcessBlock/ProcessBlock where applicable (oops) * Add FIXMEs and TODOs for Piotr plus touchups * Murder the caterpillar (for real this time) Had a bug due to not converting ancestralDBEntry to NormalDBEntry * Checksum optimization, some cleanup * Smaller HashToCurve cache * Add blockheight to every encoder * Resurrect caterpillar (prev commit); rename encode/decode * Fix Fixmes * Bump the chunk size to 800MB * Re-murder caterpillar * Revert "Re-murder caterpillar" This reverts commit 1cdf31b. * Increase max messaage payload for hypersync * Revert "Re-murder caterpillar" This reverts commit 1cdf31b. * Updaqte memtablesize for badger * Update memtablesize * Add functionality needed to support Rosetta hypersync * Set badger memtable size to 4gb * Remove blockheight from encode/decode * Fix that dirty race condition in operationChannel; add block height to ancestral cache * Add performance * Fix shift bug * Update memtablesize for badger and fix hypersync restart issue * Add a lot of error handling; add status updates and initial snapshot recovery * Fix repost prefix overwrite * Remove broken snapshot * Config param rename * Fix random ordering of NFTBidEntry in UtxoOps * Stupido index * Fix NFT royalties encoder * Add snapshot block height node param * Fix flag * Add status db * Add log * glog global params * Add global params log * Pause miner and add some test logging for global params * Revert global params logging * Mine only when fully current * Add flags for timer * More concurrency control; prelude to snapshot recovery; log cooolors * Reorganize snapshot code * Fix green * Better logging * Better logging * Small adjustments * Fix * Another fix * Faster delete db * Add faster node erase * Logs and stop address relayer * Remove division bug * More seemless restart * Fix overlap, better logs * nit * Snapshot recovery mode * Fix logging; restart on invalid checksum * Encoder migration, add StatePrefixToDeSoEncoder map * Add encoder types and versions * Add migration to checksum computation * Proper add bytes in process chunk * Persist encoder checksums; add encoder block heights * Add migration on boot; switch migrations on snapshot process block * Fix disconnect checksum; test recovery mode; add migration tests * Fix block reward disconnect; simplify migrations; better recovery and restarts * Debug migrations; add ancestral delete * Fix nil pointer * Nice message when you close the node * Faster stops * Increase chunk size * Update restarts * Fix lib tests * Fix disconnects * Db update * Small cleanup * More comments * Fix pg db adapter * Archival mode * gofmt * More comments; automated encoder tests; fix txindex encoder * Cleanup; fix the NFT utxoop thing * Better chain state condition * Fetch blockNode by hash and fix blockHash equality * Resolve merge conflict agaaaain * Fix dao limit order fees * Fix nilptr reference; disaster condition in node; comments in snapshot * More cleanup and fix expected response * small comments * Make migration scheme utilize ForkHeight variables (reduce redundandcy) * Refactors; comments; fixes * Update comments * Fix * Add persistent flags * Comments, better version encoding * Run flags * The diamondhands holy pass is complete. * Add missing flag Co-authored-by: p <p> Co-authored-by: maebeam <maebeam0@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: root <root@ip-172-31-48-134.ec2.internal> * Fix postgres null balance entry. (deso-protocol#345) * Fix postgres null balance entry. * Check for nil in NewBalanceEntry. * Support DAOCoinLimitOrder cancels for derived keys (deso-protocol#347) * Fix regtest (deso-protocol#348) * Fix regtest * gofmt * Make --archival-mode true by default for now * Add a DB version to auto-change the datadir when major updates are made (deso-protocol#349) * [stable] Release 2.1.3 * Move float -> uint256 scaling calculation to a general purpose function (deso-protocol#350) * Move float -> uint256 scaling calculation to a general purpose function * add constants for NotGraylisted and NotBlacklisted (deso-protocol#196) * Diamondhands/fix hard fork issues (deso-protocol#351) * Fix nil TransactionSpendingLimitTracker issue * Fix nil check in logging of TransactionSpendingLimitTracker * Add nil checks for transaction spending limit maps Co-authored-by: lazynina <lazynina84@gmail.com> * Move Bitcoin RBF check to core * Refactor hypersync flags to be able to force blocksync or hypersync * Set mainnet block heings for DAODAO hard fork (deso-protocol#353) Hard fork date: Wednesday at 2pm PT (~8.5 days from now) Computing 1 week of blocks: * Actual block diff for 7 days from block explorer: - 128309 (today) * Mon May 09 2022 13:55:28 GMT-0700 (Pacific Daylight Time) - 126339 (7 days ago) * Mon May 02 2022 13:42:51 GMT-0700 (Pacific Daylight Time) - = 1970 blocks * From calculations: - 288 * 7 - 2016 * Sanity-check: - (2016-1970) = 46 blocks diff - ~2.3% difference = 3.8 hours (not a big deal) - This is actually good because it means that we have some buffer Computing blocks from now until hard fork * 128309 - Mon May 09 2022 13:55:28 GMT-0700 (Pacific Daylight Time) * Add precisely 9 days of blocks: - 9 * 288 = 2592 blocks - 128309 + 2592 = 130901 * [stable] Release 2.2.0 * [stable] Release 2.2.1 Bumping release to fix a CI issue * Allow SignTransactionBytes to sign derived or non-derived key * Remove txindex reliance on snapshotting * Expose postgres db to other services * Fix bug in limit order logic around small orders We discovered a minor bug whereby orders could be submitted that are so small that they can be filled without the other side of the trade paying anything. While this is generally harmless, some error-checking logic made it so that new orders would be rejected if the order on the top of the book had this issue. This change resolves this issue and adds thorough testing around it. Added test case to reproduce bug Fix bug and add comprehensive test cases Fix test issue * [stable] Release 2.2.2 * Fix failing postgres. (deso-protocol#356) * Beef up limit order test case * Fix RawDecodeWithoutMetadata for NFTBidEntryBundle (deso-protocol#357) * Revert "Fix RawDecodeWithoutMetadata for NFTBidEntryBundle (deso-protocol#357)" This reverts commit beb8e6c. This commit was breaking the hypersync checksum. To reproduce, simply sync a testnet node. We should fix this before we deploy this change to prod nodes. * Postgres: fix get block rewards for public key (deso-protocol#360) * Fix RawDecodeWithoutMetadata for NFTBidEntryBundle * Fix query to get block rewards for public key * Revert fix to raw decode without metadata for nft bid entry bundle * [stable] Release 2.2.3 * Skip broken txns in the mempool to unblock nodes * Update NinjaNode => VoSocial App (deso-protocol#359) * Move derived key spending limits check outside of verifySignatures conditional (deso-protocol#364) * Move derived key spending limits check outside of verifySignatures conditional * Fix linter error * Update nodes.go (deso-protocol#367) * Fix miss of prevOrder in UtxoOps (deso-protocol#370) * Fix restart (deso-protocol#369) * update node name from CloutFeed to Desofy (deso-protocol#374) * Allow Eth signatures for derived key access signatures (deso-protocol#371) * Test metamask and eth signatures * Accept Eth signatures while verifying accessSignatures * fix miner * Add a public key check * gofmt lol * Fix data hash * Add blockheight * gofmt * Small speed-up to skip db fetching orders already in view. * Set block height for order book optimization * Fix miner arguments * Set fork heights for eth-sign and order matching optimization * Fix gofmt * Bump testnet heights * [stable] Release 2.2.4 * Incorporate order book speedup block heights. * [stable] Release 2.2.5 * fix conflicts * fix log msg for sqs Co-authored-by: maebeam <maebeam0@gmail.com> Co-authored-by: Erwin Willems <github@reflexlight.nl> Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: Piotr Nojszewski <piotrek.nojszewski@gmail.com> Co-authored-by: Hunter Paulson <hpaulson@seismiccore.com> Co-authored-by: Camden Foucht <Foucht@gmail.com> Co-authored-by: thoughtlesslabs <john@thoughtlesslabs.com> Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: Matt Pitts <mattapitts@gmail.com> Co-authored-by: diamondhands0 <81935176+diamondhands0@users.noreply.github.com> Co-authored-by: kanshi <andrzej@yexperiment.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com> Co-authored-by: BitClout.com/u/tijn <69529928+tijno@users.noreply.github.com> Co-authored-by: superzordon <88362450+superzordon@users.noreply.github.com> Co-authored-by: Juan Carlos del Valle <jc.ekinox@gmail.com> Co-authored-by: elmasapp <96642041+elmasapp@users.noreply.github.com> Co-authored-by: Don Hardman <59067742+donhardman@users.noreply.github.com> Co-authored-by: redpartyhat <81658108+redpartyhat@users.noreply.github.com> Co-authored-by: Michel Majdalani <43356588+MichelMajdalani@users.noreply.github.com> Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> Co-authored-by: iamsofonias <99746187+iamsofonias@users.noreply.github.com> Co-authored-by: lazynina <lazynina84@gmail.com> Co-authored-by: root <root@ip-172-31-48-134.ec2.internal> Co-authored-by: Ribal Al Hatem <33269010+RHatem@users.noreply.github.com> Co-authored-by: mattfoley8 <isaac@deso.org>
piratekev
added a commit
to TheBitgram/core
that referenced
this pull request
Sep 1, 2022
* Use nil blockhashex on mempool transactions (deso-protocol#140) * Add a list of nodes maintained by the core developer team (deso-protocol#145) Co-authored-by: Erwin Willems <github@reflexlight.nl> Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> * Limit mempool txn size to half of block size * Fix BTC buy check * Increase logging on BlockCypherCheckBitcoinDoubleSpend * Simpler derived keys (deso-protocol#148) * Simplify * Fix Co-authored-by: p <p> * Added additional nodes (deso-protocol#149) * add new node (deso-protocol#152) * Missing comma on newest node (deso-protocol#157) * Add DeversoNode (deso-protocol#156) * add double spend detection source in error message (deso-protocol#159) * add double spend detection source in error message * Add txn hash to double-spend error logging Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Add support for nft transfer notifications (deso-protocol#158) * Add support for nft transfer notifications * add trailing comma in nodes.go * Move vendored libraries to separate repos (deso-protocol#160) * [stable] Release 1.2.8 * update GetPostsPaginatedForPublicKeyOrderedByTimestamp to take in nftsRequired boolean (deso-protocol#161) * Adding node (deso-protocol#163) * Split block view and tests (deso-protocol#164) * expose the prefix for the index from public key to deso balance (deso-protocol#168) * expose the prefix for the index from public key to deso balance * expose prefix for creator deso locked * Factor the double-spend check into its own function (deso-protocol#165) * Factor the double-spend check into its own function * return boolean indicating if error is from double spend detection or not (deso-protocol#167) * return boolean indicating if error is from double spend detection or not * flip return values order Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Ln/fix minor disconnect bid issue (deso-protocol#169) * [stable] Release 1.2.9 * Buy Now NFTs with Buy Now Price (FORKING CHANGE) (deso-protocol#166) * Refactor connectAcceptNFTBid to move logic that handles transferring the NFT, computing royalties and payouts, and executing payouts so it can be reused in connectNFTBid to support Buy now NFTs * update disconnect logic for nft bid to account for buy now NFTs * update tests and disconnect logic for NFT bids * finish buy now test and add more comments around test * Add NFTBidCreatorPublicKey, NFTBidBidderPublicKey, NFTBidCreatorRoyaltyNanos to UtxoOperation for NFTBid transactions * remove unused BidderInputs from NFTBidMetadata, add comments above IsBuyNowKey * update notifications for NFT bid on buy now nft * remove commented out line * clean up comments * add block height to trigger Buy Now NFTs * fix tests involving buy now nfts * address RPH feedback * address a lot of DH feedback * Update to have a separate price for Buy Now NFTs instead of using MinBidAmountNanos * force Buy Now price to be greater than min bid amount * check in current progress on tests * finish out testing of Buy now NFT with separate buy now price * submitting a bid on serial number 0 will not trigger a buy now operation. You must specifically bid on a serial number for a buy now to occur * resolve merge conflicts with main * resolve conflicts in block_view_nft.go * set rosetta fields appropriate in helpConnectNFTSold * remove rosetta fields from initial construction * diamondhands review, clean up, fix a couple minor bugs * Moved the _helpConnectNFTSold block to closer to where it was previously to make reviewing easier. * Caught an error in CreateAcceptNFTBid that wasn't being caught before. * Fixed the ordering of adding to the utxoOperations list in _helpConnectNFTSold. The ops were being added in the wrong order before. In particular, we were doing: - _connectBasicTransfer - Spending the BidderInputs - Adding the royalties But the UtxoOps were `[bidder inputs, add royalties, connect basic transfer]`, when they should have been `[connect basic transfer, bidder inputs, add royalties]`, i.e. same order as the operations we performed. * Fixed some comments here or there Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * DAO coins (deso-protocol#174) * Refactor connectAcceptNFTBid to move logic that handles transferring the NFT, computing royalties and payouts, and executing payouts so it can be reused in connectNFTBid to support Buy now NFTs * update disconnect logic for nft bid to account for buy now NFTs * update tests and disconnect logic for NFT bids * finish buy now test and add more comments around test * Add NFTBidCreatorPublicKey, NFTBidBidderPublicKey, NFTBidCreatorRoyaltyNanos to UtxoOperation for NFTBid transactions * remove unused BidderInputs from NFTBidMetadata, add comments above IsBuyNowKey * update notifications for NFT bid on buy now nft * remove commented out line * clean up comments * add block height to trigger Buy Now NFTs * fix tests involving buy now nfts * address RPH feedback * address a lot of DH feedback * Update to have a separate price for Buy Now NFTs instead of using MinBidAmountNanos * force Buy Now price to be greater than min bid amount * check in current progress on tests * finish out testing of Buy now NFT with separate buy now price * submitting a bid on serial number 0 will not trigger a buy now operation. You must specifically bid on a serial number for a buy now to occur * resolve merge conflicts with main * resolve conflicts in block_view_nft.go * set rosetta fields appropriate in helpConnectNFTSold * remove rosetta fields from initial construction * Initial check-in of DAO Coin PR * add migration file * add more comments and make some code a little cleaner * address piotr feedback * address more Piotr feedback and add network tests * update HasPurchased in GetHoldings and GetHolders * fix indent on OperationTypeDAOCoinTransfer * add transfer restrictions to DAO coins * update postgress accordingly{ * update postgres handling * fix rule error capitalization * diamondhands review, clean up, fix a couple minor bugs * Moved the _helpConnectNFTSold block to closer to where it was previously to make reviewing easier. * Caught an error in CreateAcceptNFTBid that wasn't being caught before. * Fixed the ordering of adding to the utxoOperations list in _helpConnectNFTSold. The ops were being added in the wrong order before. In particular, we were doing: - _connectBasicTransfer - Spending the BidderInputs - Adding the royalties But the UtxoOps were `[bidder inputs, add royalties, connect basic transfer]`, when they should have been `[connect basic transfer, bidder inputs, add royalties]`, i.e. same order as the operations we performed. * Fixed some comments here or there * Fix merge conflict with test * Merbe _connectCreatorCoinTransfer and _connectDAOCoinTransfer * [REQUIRES RESYNC] Cleanup and rename CoinEntry -> CreatorCoinEntry * Move HelpConnectCoinTransfer * Add missing import * [REQUIRES RESYNC] Switch to uint256 for coin balances * In txns: - DAOCoinMetadata * CoinsToMintNanos * CoinsToBurnNanos - DAOCoinTransferMetadata * DAOCoinToTransferNanos In db structs: - CoinEntry * CoinsInCirculation * Didn't need CoinWatermarkNanos because CreatorCoins don't go over uint64 - BalanceEntry * BalanceNanos All tests passing * Add overflow tests for DAO coins * Nit: Add error logging for balance lookup Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Ln/nft splits (deso-protocol#173) * Refactor connectAcceptNFTBid to move logic that handles transferring the NFT, computing royalties and payouts, and executing payouts so it can be reused in connectNFTBid to support Buy now NFTs * update disconnect logic for nft bid to account for buy now NFTs * update tests and disconnect logic for NFT bids * finish buy now test and add more comments around test * Add NFTBidCreatorPublicKey, NFTBidBidderPublicKey, NFTBidCreatorRoyaltyNanos to UtxoOperation for NFTBid transactions * remove unused BidderInputs from NFTBidMetadata, add comments above IsBuyNowKey * update notifications for NFT bid on buy now nft * remove commented out line * clean up comments * add block height to trigger Buy Now NFTs * fix tests involving buy now nfts * address RPH feedback * address a lot of DH feedback * Update to have a separate price for Buy Now NFTs instead of using MinBidAmountNanos * force Buy Now price to be greater than min bid amount * check in current progress on tests * finish out testing of Buy now NFT with separate buy now price * submitting a bid on serial number 0 will not trigger a buy now operation. You must specifically bid on a serial number for a buy now to occur * resolve merge conflicts with main * resolve conflicts in block_view_nft.go * check in current progress on NFT splits * start test of nft splits * add IsDeleted function to pkid entries * add more error case testing * add comments, resolve TODOs, and check for overflow * set rosetta fields appropriate in helpConnectNFTSold * remove rosetta fields from initial construction * refactor construction of additional royalties to reduce code duplication * go fmt project * Review splits. Ready to ship! * Made custom deterministic serializers rather than using gob * Changed interface to use public key, hiding PKID * Added SwapIdentity to the test on splits. Looks good * Fixed a minor overflow bug * TODO: Add a test for overflows * add more checks for overflow and add more testing for overflow * add one more test on overflow for good measure Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Fix bug introduced by NFT splits (deso-protocol#176) * Add Tunel node (deso-protocol#177) * Check CreatorPKID instead of HODLerPKID when iterating over utxoview (deso-protocol#178) * Move blockheights into DeSoParams and set values accordingly (deso-protocol#171) * Move blockheights into DeSoParams and set values accordingly * fix test * remove unnecessary update to set blockheight * resolve conflicts with main * fix alignment and remove original constants for block heights * go fmt constants.go * fix BuyNowAndNFTSplitsBlockHeight usage in extractAdditionalRoyaltyMap * use params.DAOCoinBlockHeight * use params.BuyNowAndNFTSplitsBlockHeight * Embed fork heights in a struct * Make testnet heights equal to previous values and fix broken test Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Messages V3 (deso-protocol#182) * Finish connect/disconnect logic * Add requests * Add Party Entry, Start Connect/Disconnect * Finish connecting party messages; all core logic done * Small rename * Fix messaging key inconsistencies; general clean * Cleanup code; Add tests for messaging keys * Fix weird indents * Add initial tests for message parties * Finish testing * Thoroughly tested and some more cleanup * Add support for get/send backend endpoints * Update backend endpoint * Add block height * Lang * Finish core & add tests * Finish testing messaging keys; hard part done, now onto private message tests * Cleanup test comments * Fix private message connect/disconnect; cleanup * Add more tests * Fix setters/getters for message entries * All done & tested * Final cleanup * Add network tests * Fix isRecipient * Simplify messaging key entry * Simplify messaging key entry (deso-protocol#181) * Unencrypted groups support * Fork heights * Clean version code * Nits * Upgrade some comments * Clean up messages v3 - Fixed isDeleted bug in the fetching of group messages - Rename everything to be group-oriented: * MessagingKey -> MessagingGroupKey * MessagingKeyEntry -> MessagingGroupEntry * recipient -> member * MessagingRecipient -> MessagingGroupMember - Refactor some db stuff. Mainly added GroupOwnerPublicKey to the MessagingGroupMember, and cleaned up the back-reference there. * Test fix * Final passerinio * Good luck pass; and prefix fix * Unfix prefix Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Fix instances where a pointer is taken of a map key iterator (deso-protocol#183) * Fix another iterator pointer issue (deso-protocol#184) * Fix stubborn NFT splits bug due to reordering of royalties in map (deso-protocol#188) * [stable] Release 2.0.0 (deso-protocol#190) * Set block height triggers for DAO Coins, NFTs v2, and Messages v3 * [stable] Release 2.0.1 * Fix sql statements for DAOCoin/Nft (deso-protocol#193) * Add new constant to constants file to represent the NodeSource map key in Extra Data (deso-protocol#191) * Adding entre's node. (deso-protocol#179) * nft and dao coin notifications after fork (deso-protocol#185) * add support for DAO coin and nft royalty notifications * updates to better support notifications on the frontend * fix issue with pointers in for loops * merge in main and copy iterator * fix creator pub key * Fix creator public key in nft bid * skip affected pub key if royalty amount is 0 * simplify owner public key in NFT bid case * simplify creator public key in Accept NFT Bid case * add deso royalty map to utxo * make the NFT bid case mirror accept NFT bid * [stable] Release 2.0.2 * Update Postgres handling of DAO Coins and NFTs (deso-protocol#194) * [stable] Release 2.0.3 * Add a SQL adapter for BlockHash that makes it work with bun (deso-protocol#204) This will allow us to upgrade to Postgres v14 bu using the bun library rather than the super old go-pg library, which is still stuck at v10 * fix DAO coin postgres issues (deso-protocol#208) * [stable] Release 2.0.4 (deso-protocol#209) * create maps for additional royalties when converting to/from PGPostEntry before copying values into it (deso-protocol#212) * create maps for additional royalties when converting to/from PGPostEntry before copying values into it * Add debugging to error path * Update encoding of pkid in AdditionalNFTRoyaltiesToCoinsBasisPoints * Add group messaging support * Add messaging group tables * Fix small bug in down of migration Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Return error if error during txindex db update (deso-protocol#220) * Fix Issue with ParentStakeID set when modifying Post (deso-protocol#222) * move ComputeMetadata to a switch-case and glog errors for Like and SubmitPost Txns * don't return an error from ComputeTransactionMetadata, just glog it * Create accept bid for buynow when create bid in postgres (deso-protocol#226) * Create accept bid for buynow when create bid in postgres * Update lib/postgres.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * add AcceptedBlockHeight to NFTBidEntry (REQUIRES RESYNC) (deso-protocol#224) * add AcceptedBlockHeight to NFTBidEntry * Set Accepted on PGNFTBid is AcceptedBlockHeight is not nil * only copy AcceptedBlockHeight if not nil * add txindex metadata for Burns and Accept NFT Transfers (deso-protocol#228) * Update nodes.go (deso-protocol#202) * Update nodes.go * Update nodes.go * Add upcoming overclout node (deso-protocol#219) * Add upcoming overclout node * Update lib/nodes.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Fix get-all-messaging-keys & add base point test (deso-protocol#217) * add remaining UtxoTypes to UtxoType.String() (deso-protocol#229) * [stable] Release 2.0.6 (deso-protocol#231) * ch pgBidNft is not nil (deso-protocol#232) * [stable] Release 2.0.7 (deso-protocol#233) * Ln/spending limits (deso-protocol#216) * Fix profile vulnerability. (#1) * lower height * Add transaction spending limit for derived keys save current progress on derived key spending limit add more testing, remove usage of pointers in limit map keys * separate SET and TRACK block heights for derived key transaction spending limits * remove weird test * address Piotr feedback - encoders/decoders for map keys, add comments, refactor inline functions * go fmt project * add TransactionSpendingLimitTracker and Memo to PGDerivedKey * updates to converting DAO/NFT/CC operations to string for backend usage * compare operation limit keys by hex encoding * add more tests * diamondhands edits * merge in main * Add ZeroBlockHash and use ZeroBlockHash and ZeroPKID in map keys * diamondhands touchups * set regtest forkheights to 0 * add test that derived key can sign its own transactions and spend its own money * Address fixmes * rebuild * add test case for minting NFT * reduce nested if statements in block_view.go * Update lib/block_view_types.go Co-authored-by: redpartyhat <81658108+redpartyhat@users.noreply.github.com> Co-authored-by: maebeam <maebeam0@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Extra Data on Posts, NFTs, Messages, Messaging Groups, and Derived Keys (deso-protocol#213) * Add ProfileExtraData to ProfileEntry struct * add test to ensure extra data comes through. update calls to CreateUpdateProfileTxn, add postgres support * Update lib/block_view_profile_test.go * Review and cleanup * Add one more fixme * update profile test for extra data * ExtraData on NFT / Message / GroupMessagingKey / DerivedKey entries (deso-protocol#221) * Return error if error during txindex db update (deso-protocol#220) * Add extra data to NFT, Message, GroupMessagingKey, and DerivedKey entries Return error if error during txindex db update (deso-protocol#220) add tests for adding extra data on NFT, private message, messaging group, and derived key entries add comment on extra data * Fix Issue with ParentStakeID set when modifying Post (deso-protocol#222) * move ComputeMetadata to a switch-case and glog errors for Like and SubmitPost Txns * don't return an error from ComputeTransactionMetadata, just glog it * Create accept bid for buynow when create bid in postgres (deso-protocol#226) * Create accept bid for buynow when create bid in postgres * Update lib/postgres.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * add AcceptedBlockHeight to NFTBidEntry (REQUIRES RESYNC) (deso-protocol#224) * add AcceptedBlockHeight to NFTBidEntry * Set Accepted on PGNFTBid is AcceptedBlockHeight is not nil * only copy AcceptedBlockHeight if not nil * add txindex metadata for Burns and Accept NFT Transfers (deso-protocol#228) * Update nodes.go (deso-protocol#202) * Update nodes.go * Update nodes.go * Add upcoming overclout node (deso-protocol#219) * Add upcoming overclout node * Update lib/nodes.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Fix get-all-messaging-keys & add base point test (deso-protocol#217) * add remaining UtxoTypes to UtxoType.String() (deso-protocol#229) * diamondhands touchups with some FIXMEs for @lazynina * delete fixmes about deleting protected keys * add test for encoding/decoding message entries for backwards compatibility * add test for encoding/decoding messaging group entries for backwards compatibility Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> Co-authored-by: elmasapp <96642041+elmasapp@users.noreply.github.com> Co-authored-by: Don Hardman <59067742+donhardman@users.noreply.github.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Update lib/blockchain.go Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> Co-authored-by: elmasapp <96642041+elmasapp@users.noreply.github.com> Co-authored-by: Don Hardman <59067742+donhardman@users.noreply.github.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com> * Gofmt the whole repo * simplify extra data handling in CreateAuthorizeDerivedKey (deso-protocol#235) * operationtype typo in disconnect (deso-protocol#238) * Fix postgres down migrations. (deso-protocol#276) * Make derived key txn construction accept a spending limit hex, not object (deso-protocol#297) * Make derived key txn construction accept a spending limit hex, not object * Update lib/block_view_derived_key.go * add exists declaration back. Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Use NewPKID to get bidder PKID if bidder's profile is not found (deso-protocol#262) * Feature/add dao coin limit order transaction (deso-protocol#307) * Add DAO Coin Limit Order network skeleton. (deso-protocol#239) * Add DAO Coin Limit Order constants. (deso-protocol#240) * Add DAO Coin Limit Order mempool skeleton. (deso-protocol#243) * Add DAO Coin Limit Order mempool skeleton. * Restore removed imports. * Side-step unused var error. * Add DAO Coin Limit Order blockchain skeleton. (deso-protocol#242) * Add DAO Coin Limit Order blockchain skeleton. * Restore removed imports. * Add DAO Coin Limit Order block view skeleton. (deso-protocol#241) * Add DAO Coin Limit Order block view skeleton. * Restore removed import. * Side-step unused var error. * Add DAO Coin Limit Order block view flush skeleton. (deso-protocol#244) * Add DAO coin limit order db prefixes. (deso-protocol#245) * Mf/add dao coin limit order key builders (deso-protocol#246) * Add DAO coin limit order key builders. * Use uint32s for enums. * Remove unnecessary line. * Mf/add dao coin limit order db utils (deso-protocol#247) * Add DAO coin limit order db utils. * Add FromBytes skeleton method. * Fix FromBytes operation happens in-place. * Mf/add dao coin limit order postgres skeleton (deso-protocol#248) * Add DAO coin limit order postgres skeleton. * Fix spacing/tabs. * Fix method call capitalization. * Convert uint256 types to strings. * Add DAO coin limit order network metadata. (deso-protocol#249) * Revert DAO coin limit order enums to uint8s. (deso-protocol#250) * Add DAO coin limit order connect transaction logic. (deso-protocol#259) * Mf/add dao coin limit order txn testing (deso-protocol#263) * Switch DAO coin order limit quantity to pointer. (deso-protocol#265) * Add more DAO coin limit order testing scenarios. (deso-protocol#267) * Convert DAO coin limit order price from BigFloat to Uint256. (deso-protocol#266) * Convert DAO coin price to uint256. * Complete conversion of DAO limit order price to uint256. * Use Lt, Gt instead of Cmp. * Start inverting price comparison logic. * add function to calculate total cost of an order (deso-protocol#268) * add function to calculate total cost of an order * Remove Conditional from DBGetMatchingDAOCoinBidOrders that didn't apply, Fix IsBetterAskThan and IsBetterBidThan (deso-protocol#269) * add check that order is better than last seen order when getting next orders to fill, fix txn metadata for m1's txn in test * use total cost calculating function to accurately get total cost of orders in blockchain.go, set prefixKey correctly for DBGetMatchingDAOCoinAskOrders Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Mf/track dao coin limit order utxos (deso-protocol#270) * Track DAO coin limit order UTXOs. * Add disconnect logic. * Fix failing test. * Update lib/block_view_dao_coin_limit_order.go * Add DAO coin limit order postgres structs. (deso-protocol#271) * Add DAO coin limit order postgres structs. * Update dao coin limit order postgres migration. * Fill out LeftPaddedHexToUint256. * Add DAO coin limit order postgres tests. (deso-protocol#273) * Remove logging before flags error. (deso-protocol#274) * Fix DAO coin down migration. (deso-protocol#275) * Fix some postgres testing issues. (deso-protocol#277) * Add additional DAO coin limit order test and resolve bugs it surfaces. (deso-protocol#278) * Add additional DAO coin limit order testing. * Refactor block view to use db adapter for DAO coin limit orders. * Set up multi-order test. * Give m0 more initial in testing. * Fix failing txn spend amt test. * Fix updating order's quantity bug. * Add helper function to query for all orders. * Fix by transactor PKID prefix bug. * Mf/better dao coin limit order var naming (deso-protocol#279) * Refactor DAO coin limit order variable naming. * Rename order to matchingOrder. * Test DAO coin limit order total cost doesnt overflow validation. (deso-protocol#280) * Mf/allow dao coin limit order cancellations (deso-protocol#281) * Allow DAO coin limit order cancellations. * Fix failing postgres test. * Add more comments. * Add failing test to show no transactorPKID in index bug. * Update helper utils to new order schema. * Checkpoint. Store progress updating connect logic. * Another checkpoint. Stash progress in connect logic. * Commit checkpoint. Updated all of block view. * Start rebuilding tests. * Checkpoint. Continue updating tests. * Add back endpoint to query open orders by transactor PKID. * Add function to get all limit orders for a DAO coin pair. * Clean up postgres. * Convert price from uint256 to big.Float. * Add uint256 ScaledPrice field. * Continue turning back-on more tests. * Fix more tests. * Checkpoint. Continue fighting tests. * Continue fixing tests. * Fix last two tests. * Revised a lot of things, but mainly _connect logic. Let's have a meeting to discuss. * Convert PKID vars to PublicKeys. * Start fixing tests. * Fix tests by removing reflect.DeepEqual. * Switch buying and selling DAO coins back to PKIDs. * Fix a few more failing tests. * Must compare transactor buying qty to matching selling qty. * Continue fixing tests. * Get cancellation tests passing. * Only update orders within the same block height. * Fix small postgres bug. * Ln/dao coin limit order txindex (deso-protocol#283) * initial check in for txindex for dao coin limit orders * Use new struct to track fulfilled orders * update naming convention * Mf/add safe math uint256 utils (deso-protocol#285) * Add SafeMathUint256 utils. * Convert some DAO coin limit order math to safe math. * Save me some whitespace. * Mf/add safe math uint64 utils (deso-protocol#287) * Add SafeMathUint256 utils. * Convert some DAO coin limit order math to safe math. * Save me some whitespace. * Add SafeMathUint64 utils. * Update DAOCoinLimitOrderMetadata type so it can be serialized to json (deso-protocol#286) * Update DAOCoinLimitOrderMetadata type so it can be serialized to json * Re-autoformat comments * Address feedback on sorting slices * Address feedback in connect transaction function * Mf/add dao coin transfer restriction validations (deso-protocol#288) * Add DAO coin limit order transfer restriction validations. * Make a bit cleaner. * Keep full error message. * Add UTXO view API DAO coin limit order getters. (deso-protocol#290) * Add UTXO view API DAO coin limit order getters. * Make merging database and UTXO values more consistent. * Only compute order map key once. * Ln/dao coin limit order disconnect (deso-protocol#289) * Add temporary postgres DAO coin limit order solution. (deso-protocol#292) * Initial check in for spending limits on DAO Coin Limit Orders (deso-protocol#293) * Initial check in for spending limits on DAO Coin Limit Orders * add comments, rename key to DAOCoinLimitOrderLimitKey, initialize map in copy, add tests for spending limit on dao coin limit orders * Add additional DAO coin limit order tests. (deso-protocol#291) * Ln/dao coin limit order cancellation validations (deso-protocol#294) * Add additional DAO coin limit order tests. * fix infinite loop with last seen order, fix validations around cancellations, update tests Co-authored-by: mattfoley8 <isaac@deso.org> * Address fixmes: add sanity checks and comments. Add isDeleted checks and new RuleErrors (deso-protocol#295) * Ln/dao coin limit order loop till filled (deso-protocol#296) * Loop until order is filled, fix db utils for getting next order, add more tests, fix copy DAOCoinLimitOrderEntry to not return error * leave comment in test for diamondhands * add balance checks in latest test * Diamondhands touch-ups * Add postgres CI whitelisted testing. (deso-protocol#298) * Add postgres CI whitelisted testing. * Run migrations before testing. * Map postgres port to localhost. * Empty commit to test caching dependencies. * Test migration rollbacks as well. * Add newline at EOF. * Mf/add additional dao coin order book tests (deso-protocol#299) * Add additional DAO coin order book tests. * Dont say bid/ask. * Add go-fmt to CI. (deso-protocol#300) * Add go-fmt to CI. * Fix failing gofmt error. * Address DH fixmes (deso-protocol#301) * Add another test. (deso-protocol#302) * Fix inputs from transactor to account for fee AND DESO spent purchasing DAO Coins, fix postgres, and add FeeNanos to txn metadata (deso-protocol#303) * Fix inputs from transactor to account for fee AND DESO spent purchasing DAO Coins, fix postgres, and add FeeNanos to txn metadata * Clone quantity to be safe * go fmt blockchain.go * always write utxo ops to badger (deso-protocol#304) * always write utxo ops to badger * run disconnect for pg * fix table name for dao coin limit order metadata * Add local postgres testing setup. (deso-protocol#305) * Make derived key txn construction accept a spending limit hex, not object (deso-protocol#297) (deso-protocol#306) * Make derived key txn construction accept a spending limit hex, not object * Update lib/block_view_derived_key.go * add exists declaration back. Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: diamondhands0 <81935176+diamondhands0@users.noreply.github.com> * Add fee nanos to the to bytes and from bytes logic for DAO Coin Limit Orders (deso-protocol#308) * Error if matching own order. (deso-protocol#309) * Make API getter functions public (deso-protocol#310) * Mf/add operation type column (deso-protocol#311) * Rename quantity to buy -> quantity to fill. * Checkpoint passing tests. * Get postgres tests working. * Add helper function and tests. * Continue adding tests for operation type quantity math. * Complete BID-BID calculation logic. * Build out ask-ask case. * Use named return values instead of a struct. * Allow for param specifying different transactor quantity to fill. * Implement logic in db_utils. * Refactor calculate test into own test. * Update blockchain.go w/ new quantity calculation. * Throw error if updating order w/ different operation type. * Update connection logic. * Fix failing derived key test. * Document off-by-one bug. * Add check on FeeNanos in DAO Coin Limit Order connect logic * Check for overflow before mutliplying fee nanos * revert changes to UpdateGlobalParams txn construction * Update w/ PR review feedback. * Make price calculators var naming a bit more consistent. * Remove DAO prefix from filled order struct qty fields. * Empty commit to trigger CI. * Fix off-by-one bug. * Fix floating point issues in tests, and implement fixed-point conversion scheme * Some touch-ups to the order matching logic. * Add a FIXME for an ultimate sanity-check in _connectDAOCoinLimitOrder * Add money printer prevention validation. * WIP - fix prevBalances, but sanity-check fails * Fix prevBalances bug and fix sanity-check after * Resolve merge conflicts. * Fix tests with a workaround on fee computation * diamondhands holy tidy pass * Fix failing postgres test. Co-authored-by: lazynina <lazynina84@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * Add DAO coin limit order blockheight validation. (deso-protocol#314) * Add DAO coin limit order blockheight validation. * Empty commit to retrigger CI. * Set DAO coin limit order block height in derived key test. * Fix get next orders to fill qty check. (deso-protocol#315) * Add some cosmetic updates. (deso-protocol#316) * don't return error from functions that don't generate errors (deso-protocol#318) * TotalInput and TotalOutput should consider inputs and outputs to/from bidders and difference should equal fee (deso-protocol#317) * TotalInput and TotalOutput should consider inputs and outputs to/from bidders and difference should equal fee * Update lib/block_view_dao_coin_limit_order.go Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> * Update lib/block_view_dao_coin_limit_order.go Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> * Test submitting ASK order and matching existing BID. (deso-protocol#319) Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: iamsofonias <99746187+iamsofonias@users.noreply.github.com> Co-authored-by: diamondhands0 <81935176+diamondhands0@users.noreply.github.com> Co-authored-by: lazynina <lazynina84@gmail.com> * Mf/additional dao coin limit order testing (deso-protocol#324) * Only copy TransactionSpendingLimitTracker if non-nil, (tsl *TransactionSpendingLimit) Copy() should return nil if tsl is nil (deso-protocol#325) * Mf/add dao coin limit order id field (deso-protocol#327) * Start adding DAO coin limit order ID field. * Add OrderID field to connect logic cancellation. * Simplify MapKey now that we have an order uuid. * Fix failing postgres tests. * Fix failing derived key test. * Fix comments / spacing. * Add OrderID to FilledOrder. * Store copy of order in prev in case of rollback. * Add additional postgres indices. * Improve parity between Postgres and Badger result set ordering. * Address review feedback pt 1. * Only store metadata CancelOrderID in bytes if present. * Use maxHash as max block hash. * Mf/dao coin order metadata optional fields (deso-protocol#329) * DAO coin order metadata optional fields. * Encode optional fields with byte count prefix. * Clean up blockchain.go branching logic. * Clean up more branching logic. * Better English in comments. * Remove TransactorPKID from primary DAO coin order book index. (deso-protocol#331) * Remove TransactorPKID from primary DAO coin order book index. * Fix failing postgres test. * Fix intermittent badger CI failure. * Remove sorting by quantity. * Remove price and blockheight from by transactor index. * Mf/test swap identity and dao coin orders (deso-protocol#330) * Test swap identity and DAO coin orders. * Test moar swapping identity + order book cases. * Make postgres tests pass. * Update m0s PKID in tests. * Add test that owner can deauthorize derived key with new transaction spending limit (deso-protocol#326) * [TESTNET HARD FORK]: Update TESTNET block heights for DAOSwap hard fork (deso-protocol#332) * [TESTNET HARD FORK]: Update TESTNET block heights for DAOSwap hard fork The following hard forks are triggering on Testnet on Wed Apr 20 @ 9am ET: ExtraDataOnEntriesBlockHeight: uint32(304087), DerivedKeySetSpendingLimitsBlockHeight: uint32(304087), DerivedKeyTrackSpendingLimitsBlockHeight: uint32(304087), DAOCoinLimitOrderBlockHeight: uint32(304087), * Update DerivedKeyTrackSpendingLimitsBlockHeight * [stable] Release 2.1.0 (deso-protocol#333) * Mf/add unused bidder inputs test (deso-protocol#334) * [stable] Release 2.1.1 (deso-protocol#335) * Mf/additional bidder input tests (deso-protocol#339) * Add tests for metadata FeeNanos. (deso-protocol#337) * Add tests for metadata FeeNanos. * Revert change to connect logic. * Mf/bidder input edge case testing (deso-protocol#338) * Bidder input edge case testing. * Add DAO<-->DAO coin test. * Fix test for including spent UTXOs. * Try increasing fee rate and retesting. * Unused UTXOs get refunded. * Use m2's public key. * Mf/add limit order metadata order type field (deso-protocol#342) * Add limit order metadata OrderType field. * Update postgres moar. * Dynamically determine order book transactor fees. * Add helper function for calculating FeeNanos. * Use uint8s for enums. * Add txn metadata validations. * Small refactor as vars used only once. * Add ImmediateOrCancel order type. * Allow zero price on IOC and FOK orders. * Allow user to specify exchange rate on IOC and FOK orders. * Add tests for market ASK orders. * Store OrderEntry.OrderType in the database. * diamondhands touchups * Require OrderType must be specified. No default value. * Refactor IsMarketOrder util function location. * Validate txn metadata. * Rename OrderType to FillType. * Update test description. * Fix failing CI. * Fix DESO balance check in DAO coin limit order The way we were validating DESO balance made it so that we could only execute a "sell" of DESO if the resulting balance was above the sell amount. This made it so that the maximum amount of DESO we could sell in an order was ~(DESO balance / 2), rather than (DESO - fee). The fix was to move the execution of _connectBasicTransfer to *after* the point where all the validations were done. This had no impact other than fixing our validation logic. * Catch an error when parsing fill type * Add more uint8 overflow checks. Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> * [stable] Release 2.1.2 (deso-protocol#344) * Hypersync (deso-protocol#346) * Add shared secret encryption/decryption * Make Encrypt Backwards Compatible * Fix comment * Minor fix * Skip unnecessary utxo deletes * Remove unnecessart balance nanos delete * Fix balance tests * Remove some unnecessary deletes * fix nft tests * Only flush PKID entries that are involved in swap identity transactions * Fix disconnect * Remove unnecessary profile deletes * Speed up balance entries and derived keys * Remove some bitcoin manager stuff * Profiles are slower but work * fix switch * try to fix test * [not safe] flush every 1k blocks * write our own encoders * Fix posts * Add hyper sync flag and fix comments * Fix... * More custom encoders * Add snapshot; Adjust DB; Add Hyper sync messages * Add snapshot to db_utils * Add snapshot everywhere; add encoders/decoders * Add checksum & ancestral records logic * Flushing ancestral records on utxo_view flush * Tests Fix * Remove delayed flush * Add utxokey encode/decode * Change dockerfile * Fix some missed fields * fix * fix * Fix appends in derivedKeyEntry * DeDupe * Fix tests * Debug snapshot; Add snapshot to all tests * Node sync fix * Initial snapshot peer messages * Snapshots are being sent * Sync works * Timing * Add ancestral records * First passing hyper sync with properly built & served ancestral records * Switch to Ristretto255 curve for checksum * DBEntry cleanup; parsing optimization * More cleanup; Change db prefixes; checksum updates * Curve games * Checksum games * Add worker pool checksum computation; checksum from block sync matches hyper sync; switch to deque in ancestral record memory * Add main and ancestral db semaphores * Fix merge conflicts * Fix checksum merge conflicts * Checksum optimization; add timer; add get snapshot validation * Another optimization * Correctly switch to block sync after state sync * Cleanup * 50% speed improvement; more cleanup * More cleanup * More comments * Document a lot * Txindex * Encoders everywhere; get rid of gob * Fix hot feed * Add encoder tests & fix postgres * Node testing framework initial * Add network bridges to node testing framework * Cleanup and document node testing; Add first tests * Add first hyper sync test * Clean formatting * Add restart tests; fmt * More tests * Some solid debugging * Debug sync past blocks after hyper sync * Test basic hyper sync followed by txindex * Txindex updates * Code review * Fix quit * wip tests passing * Slight refactor of nonexisting record thing * little wip tests passing * Fix bug around nested calls to PrepareAncestralRecordsFlush * Metadata and restarts * Rename snapshot.go:ProcessBlock to SnapshotProcessBlock * Murder the caterpillar * s/hyper-sync/hypersync * s/SnapshotProcessBlock/ProcessBlock where applicable (oops) * Add FIXMEs and TODOs for Piotr plus touchups * Murder the caterpillar (for real this time) Had a bug due to not converting ancestralDBEntry to NormalDBEntry * Checksum optimization, some cleanup * Smaller HashToCurve cache * Add blockheight to every encoder * Resurrect caterpillar (prev commit); rename encode/decode * Fix Fixmes * Bump the chunk size to 800MB * Re-murder caterpillar * Revert "Re-murder caterpillar" This reverts commit 1cdf31b. * Increase max messaage payload for hypersync * Revert "Re-murder caterpillar" This reverts commit 1cdf31b. * Updaqte memtablesize for badger * Update memtablesize * Add functionality needed to support Rosetta hypersync * Set badger memtable size to 4gb * Remove blockheight from encode/decode * Fix that dirty race condition in operationChannel; add block height to ancestral cache * Add performance * Fix shift bug * Update memtablesize for badger and fix hypersync restart issue * Add a lot of error handling; add status updates and initial snapshot recovery * Fix repost prefix overwrite * Remove broken snapshot * Config param rename * Fix random ordering of NFTBidEntry in UtxoOps * Stupido index * Fix NFT royalties encoder * Add snapshot block height node param * Fix flag * Add status db * Add log * glog global params * Add global params log * Pause miner and add some test logging for global params * Revert global params logging * Mine only when fully current * Add flags for timer * More concurrency control; prelude to snapshot recovery; log cooolors * Reorganize snapshot code * Fix green * Better logging * Better logging * Small adjustments * Fix * Another fix * Faster delete db * Add faster node erase * Logs and stop address relayer * Remove division bug * More seemless restart * Fix overlap, better logs * nit * Snapshot recovery mode * Fix logging; restart on invalid checksum * Encoder migration, add StatePrefixToDeSoEncoder map * Add encoder types and versions * Add migration to checksum computation * Proper add bytes in process chunk * Persist encoder checksums; add encoder block heights * Add migration on boot; switch migrations on snapshot process block * Fix disconnect checksum; test recovery mode; add migration tests * Fix block reward disconnect; simplify migrations; better recovery and restarts * Debug migrations; add ancestral delete * Fix nil pointer * Nice message when you close the node * Faster stops * Increase chunk size * Update restarts * Fix lib tests * Fix disconnects * Db update * Small cleanup * More comments * Fix pg db adapter * Archival mode * gofmt * More comments; automated encoder tests; fix txindex encoder * Cleanup; fix the NFT utxoop thing * Better chain state condition * Fetch blockNode by hash and fix blockHash equality * Resolve merge conflict agaaaain * Fix dao limit order fees * Fix nilptr reference; disaster condition in node; comments in snapshot * More cleanup and fix expected response * small comments * Make migration scheme utilize ForkHeight variables (reduce redundandcy) * Refactors; comments; fixes * Update comments * Fix * Add persistent flags * Comments, better version encoding * Run flags * The diamondhands holy pass is complete. * Add missing flag Co-authored-by: p <p> Co-authored-by: maebeam <maebeam0@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: root <root@ip-172-31-48-134.ec2.internal> * Fix postgres null balance entry. (deso-protocol#345) * Fix postgres null balance entry. * Check for nil in NewBalanceEntry. * Support DAOCoinLimitOrder cancels for derived keys (deso-protocol#347) * Fix regtest (deso-protocol#348) * Fix regtest * gofmt * Make --archival-mode true by default for now * Add a DB version to auto-change the datadir when major updates are made (deso-protocol#349) * [stable] Release 2.1.3 * Move float -> uint256 scaling calculation to a general purpose function (deso-protocol#350) * Move float -> uint256 scaling calculation to a general purpose function * add constants for NotGraylisted and NotBlacklisted (deso-protocol#196) * Diamondhands/fix hard fork issues (deso-protocol#351) * Fix nil TransactionSpendingLimitTracker issue * Fix nil check in logging of TransactionSpendingLimitTracker * Add nil checks for transaction spending limit maps Co-authored-by: lazynina <lazynina84@gmail.com> * Move Bitcoin RBF check to core * Refactor hypersync flags to be able to force blocksync or hypersync * Set mainnet block heings for DAODAO hard fork (deso-protocol#353) Hard fork date: Wednesday at 2pm PT (~8.5 days from now) Computing 1 week of blocks: * Actual block diff for 7 days from block explorer: - 128309 (today) * Mon May 09 2022 13:55:28 GMT-0700 (Pacific Daylight Time) - 126339 (7 days ago) * Mon May 02 2022 13:42:51 GMT-0700 (Pacific Daylight Time) - = 1970 blocks * From calculations: - 288 * 7 - 2016 * Sanity-check: - (2016-1970) = 46 blocks diff - ~2.3% difference = 3.8 hours (not a big deal) - This is actually good because it means that we have some buffer Computing blocks from now until hard fork * 128309 - Mon May 09 2022 13:55:28 GMT-0700 (Pacific Daylight Time) * Add precisely 9 days of blocks: - 9 * 288 = 2592 blocks - 128309 + 2592 = 130901 * [stable] Release 2.2.0 * [stable] Release 2.2.1 Bumping release to fix a CI issue * Allow SignTransactionBytes to sign derived or non-derived key * Remove txindex reliance on snapshotting * Expose postgres db to other services * Fix bug in limit order logic around small orders We discovered a minor bug whereby orders could be submitted that are so small that they can be filled without the other side of the trade paying anything. While this is generally harmless, some error-checking logic made it so that new orders would be rejected if the order on the top of the book had this issue. This change resolves this issue and adds thorough testing around it. Added test case to reproduce bug Fix bug and add comprehensive test cases Fix test issue * [stable] Release 2.2.2 * Fix failing postgres. (deso-protocol#356) * Beef up limit order test case * Fix RawDecodeWithoutMetadata for NFTBidEntryBundle (deso-protocol#357) * Revert "Fix RawDecodeWithoutMetadata for NFTBidEntryBundle (deso-protocol#357)" This reverts commit beb8e6c. This commit was breaking the hypersync checksum. To reproduce, simply sync a testnet node. We should fix this before we deploy this change to prod nodes. * Postgres: fix get block rewards for public key (deso-protocol#360) * Fix RawDecodeWithoutMetadata for NFTBidEntryBundle * Fix query to get block rewards for public key * Revert fix to raw decode without metadata for nft bid entry bundle * [stable] Release 2.2.3 * Skip broken txns in the mempool to unblock nodes * Update NinjaNode => VoSocial App (deso-protocol#359) * Move derived key spending limits check outside of verifySignatures conditional (deso-protocol#364) * Move derived key spending limits check outside of verifySignatures conditional * Fix linter error * Update nodes.go (deso-protocol#367) * Fix miss of prevOrder in UtxoOps (deso-protocol#370) * Fix restart (deso-protocol#369) * update node name from CloutFeed to Desofy (deso-protocol#374) * Allow Eth signatures for derived key access signatures (deso-protocol#371) * Test metamask and eth signatures * Accept Eth signatures while verifying accessSignatures * fix miner * Add a public key check * gofmt lol * Fix data hash * Add blockheight * gofmt * Small speed-up to skip db fetching orders already in view. * Set block height for order book optimization * Fix miner arguments * Set fork heights for eth-sign and order matching optimization * Fix gofmt * Bump testnet heights * [stable] Release 2.2.4 * Incorporate order book speedup block heights. * [stable] Release 2.2.5 * Fix Assorted Postgres Issues (deso-protocol#372) * Insert TransactionSpendingLimits in postgres as byte slice instead of TransactionSpendingLimit struct * If no bytes for spending limit, set to nil in postgres * Return initialized balance entry instead of nil from postgres if not in db * Fix linter errors * save current progress on pg fixes * Use badger for DAO Coin Limit Order and Messaging Group storage * Fix linter errors in block_view_message.go * Refactor param updater * Update block height for paramUpdater refactor * fix CI error "Could not open temp db to dump mempool" (deso-protocol#378) * protect concurrent access to memepool db * move lock to open db method * add lock to loadtxns function * what happens if we just dont dump on the timer * disable perodic memepool dumper in tests * manually load txns * manually set memepool dir * Update lib/block_view_bitcoin_test.go Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> * Revert line removed in previous PR (deso-protocol#381) * Add post2earndao node (deso-protocol#385) * Adding Pearl to the list of nodes (deso-protocol#388) * Adding Pearl to the list of nodes * Update nodes.go * replace send to sqs logic * fix deps for mempool * fix deps again * Update mempool.go * Update mempool.go * gofmt Co-authored-by: maebeam <maebeam0@gmail.com> Co-authored-by: Erwin Willems <github@reflexlight.nl> Co-authored-by: Martijn van Halen <martijn.vanhalen@gmail.com> Co-authored-by: diamondhands <diamondhands@bitcloutdev.com> Co-authored-by: Piotr Nojszewski <piotrek.nojszewski@gmail.com> Co-authored-by: Hunter Paulson <hpaulson@seismiccore.com> Co-authored-by: Camden Foucht <Foucht@gmail.com> Co-authored-by: thoughtlesslabs <john@thoughtlesslabs.com> Co-authored-by: Lazy Nina <81658138+lazynina@users.noreply.github.com> Co-authored-by: Matt Pitts <mattapitts@gmail.com> Co-authored-by: diamondhands0 <81935176+diamondhands0@users.noreply.github.com> Co-authored-by: kanshi <andrzej@yexperiment.com> Co-authored-by: Piotr Nojszewski <29924594+AeonSw4n@users.noreply.github.com> Co-authored-by: BitClout.com/u/tijn <69529928+tijno@users.noreply.github.com> Co-authored-by: superzordon <88362450+superzordon@users.noreply.github.com> Co-authored-by: Juan Carlos del Valle <jc.ekinox@gmail.com> Co-authored-by: elmasapp <96642041+elmasapp@users.noreply.github.com> Co-authored-by: Don Hardman <59067742+donhardman@users.noreply.github.com> Co-authored-by: redpartyhat <81658108+redpartyhat@users.noreply.github.com> Co-authored-by: Michel Majdalani <43356588+MichelMajdalani@users.noreply.github.com> Co-authored-by: Matt Foley <100429827+mattfoley8@users.noreply.github.com> Co-authored-by: iamsofonias <99746187+iamsofonias@users.noreply.github.com> Co-authored-by: lazynina <lazynina84@gmail.com> Co-authored-by: root <root@ip-172-31-48-134.ec2.internal> Co-authored-by: Ribal Al Hatem <33269010+RHatem@users.noreply.github.com> Co-authored-by: mattfoley8 <isaac@deso.org> Co-authored-by: Jackson Dean <jckson.dean@gmail.com> Co-authored-by: Erwin Willems <17808387+erwin-willems@users.noreply.github.com> Co-authored-by: Steven Lee <steven@yourgem.app>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.