Backport missing PRs from Bitcoin 0.13 - Part 4 #1804

codablock · 2017-12-21T19:39:04Z

Continuation of #1770

7db0ecb Test for signing messages (Andrew Chow) f90efbf Create signmessagewithprivkey rpc (Andrew)

…work items f0188f9 http: use std::move to move HTTPRequest into HTTPWorkItem (Wladimir J. van der Laan) 37b2137 http: Change boost::scoped_ptr to std::unique_ptr in HTTPRequest (Wladimir J. van der Laan) f97b410 http: Add log message when work queue is full (Wladimir J. van der Laan) 091d6e0 http: Do a pending c++11 simplification (Wladimir J. van der Laan)

…nv/mempool handling b559914 Move bloom and feerate filtering to just prior to tx sending. (Gregory Maxwell) 4578215 Return mempool queries in dependency order (Pieter Wuille) ed70683 Handle mempool requests in send loop, subject to trickle (Pieter Wuille) dc13dcd Split up and optimize transaction and block inv queues (Pieter Wuille) f2d3ba7 Eliminate TX trickle bypass, sort TX invs for privacy and priority. (Gregory Maxwell)

Bitcoin bitcoin#7840 has split the INVs to send into block and TX and completely ignores non-tx/non-block items in PushInventory. This is fine for Bitcoin, as they only use it for blocks and TXs, but we also have a lot of MN related messages which also need to be relayed.

e53e7c5 don't run ThreadMessageHandler at lowered priority (Kaz Wesley)

fa389d4 [qa] Switch to py3 (MarcoFalke)

…ix typo b3d18ba doc: Fedora build requirements, add gcc-c++ and fix typo (Warren Togami)

…ing.md b06f6a9 Fixed invalid example paths in gitian-building.md (JeremyRand)

…hmark 1953c40 More efficient bitsliced rolling Bloom filter (Pieter Wuille) aa62b68 Benchmark rolling bloom filter (Pieter Wuille)

…m UpdateCoins. c8b9248 Remove obsolete reference to CValidationState from UpdateCoins. (21E14)

Instead of fully backporting Bitcoin bitcoin#7958, we make true the default value for fThread. Otherwise the AlertNotify test case will fail as it expects the alerts to be processed synchronously.

65fee8e test: Revert fatal-ness of missing python-zmq (Wladimir J. van der Laan)

5ea4508 Autofind rpc tests --srcdir (Jonas Schnelli)

fad3366 [qa] pull-tester: Adjust comment (MarcoFalke) fafb33c [qa] Stop other nodes, even when one fails to stop (MarcoFalke) 2222dae [qa] Update README.md (MarcoFalke) fabbf6b [qa] Refactor test_framework and pull tester (MarcoFalke)

3902a29 Qt: Delay user confirmation of send (Tyler Hardin)

…ResetBlockFailureFlags 657e07e Rename ReconsiderBlock func to reflect real behavior (instagibbs) addb9d2 Remove state arg from ReconsiderBlock (instagibbs)

3e2c946 init: Move berkeleydb version reporting to wallet (Wladimir J. van der Laan)

…:WriteToDisk 0fd5997 Fix insanity of CWalletDB::WriteTx and CWalletTx::WriteToDisk (Patrick Strateman)

ccccc59 [qa] Add option --portseed to test_framework (MarcoFalke) fa494de [qa] pull-tester: Run rpc test in parallel (MarcoFalke)

32114dd bench: Add crypto hash benchmarks (Wladimir J. van der Laan)

fad60b3 [qa] Fix bip9-softforks blockstore issue (MarcoFalke)

…down should be type sig_atomic_t 3262316 fReopenDebugLog and fRequestShutdown should be type sig_atomic_t (Chirag Davé)

…on/AcceptToMemoryPool. d87b198 Remove unneeded feerate param from RelayTransaction/AcceptToMemoryPool. (Gregory Maxwell)

fa83a5d [qa] wallet: Temporarily disable salvagewallet test (MarcoFalke) fadd048 [doc] Link to clang-format in the developer notes (MarcoFalke) fa72f7d [doc] Remove outdated line from listunspent RPC help, fix typo (MarcoFalke) ac40ed7 Increase timeout waiting for pruned blk00000.dat (error10)

a68ec21 Use SipHash-2-4 for address relay selection (Pieter Wuille) 8cc9cfe Switch CTxMempool::mapTx to use a hash index for txids (Pieter Wuille) 382c871 Use SipHash-2-4 for CCoinsCache index (Pieter Wuille) 0b1295b Add SipHash-2-4 primitives to hash (Pieter Wuille)

fe80102 changing "(tests are) automatically run" to correspond to the earlier instance of "run automatically (on the build server)" (Matthew English)

…ns better. fb26bf0 CAddrMan::Deserialize handle corrupt serializations better. (Patrick Strateman)

…d less unnecessarily scary bf9266e Use Socks5ErrorString() to decode error responses from socks proxy. (Warren Togami) 94fd1d8 Make Socks5() InterruptibleRecv() timeout/failures informative. (Warren Togami) 0d9af79 SOCKS5 connecting and connected messages with -debug=net. (Warren Togami) 00678bd Make failures to connect via Socks5() more informative and less unnecessarily scary. (Warren Togami)

fad68f7 [qa] Reduce node count for some tests (MarcoFalke) fac9349 [qa] Remove hardcoded "4 nodes" from test_framework (MarcoFalke)

…t loop. 63ff57d Avoid integer division in the benchmark inner-most loop. (Gregory Maxwell)

1a8c4d5 [Doc] Add benchmarking notes (fanquake)

Throws exception otherwise.

It was always complaining about failed backups due to dashd restarting too fast. Additionally, since the recent RPC test framework backports, all tests are expected to print NOTHING to stderr, otherwise they fail.

Also call it with redirect_stderr=True in all cases were non-critical warnings are expected.

UdjinM6

Nice :)

Some comments:

with redirect_stderr introduced I think it makes sense to at least return -createwalletbackups=1 in some test(s) with redirect_stderr=True, say in wallet-hd.py, or to write a small new one to make sure that it actually complains as expected;
travis fails with

$ if [ "$PYZMQ" = "true" ]; then pip3 install pyzmq --user ; fi
The program 'pip3' is currently not installed. To run 'pip3' please ask your administrator to install the package 'python3-pip'

see inline comments below

UdjinM6 · 2017-12-21T23:56:05Z

src/rpc/misc.cpp

+            "\nCreate the signature\n"
+            + HelpExampleCli("signmessagewithprivkey", "\"privkey\" \"my message\"") +
+            "\nVerify the signature\n"
+            + HelpExampleCli("verifymessage", "\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ\" \"signature\" \"my message\"") +


should be dashified

UdjinM6 · 2017-12-22T00:35:30Z

src/validation.cpp


                CBlockIndex* pindex = (*mi).second;
-                LogPrintf("ReprocessBlocks -- %s\n", (*it).first.ToString());
+                LogPrintf("ResetBlockFailureFlags -- %s\n", (*it).first.ToString());


UdjinM6 · 2017-12-22T01:09:44Z

src/qt/transactionview.cpp

        transactionView->setSelectionMode(QAbstractItemView::ExtendedSelection);
        transactionView->setSortingEnabled(true);
-        transactionView->sortByColumn(TransactionTableModel::Status, Qt::DescendingOrder);
+        transactionView->sortByColumn(TransactionTableModel::Date, Qt::DescendingOrder);


To match 8014 the same fix should be applied to the filter in OverviewPage::SetupTransactionList

UdjinM6 · 2017-12-22T01:19:24Z

CONTRIBUTING.md

 -------------------------

-The following applies to code changes to the Dash Core project (and related projects such as libsecp256k1), and is not to be confused with overall Dash Network Protocol consensus changes.
+The following applies to code changes to the Dash Core project (and related projects such as libsecp256k1), and is not to be confused with overall Bitcoin Network Protocol consensus changes.


this piece should be reverted

UdjinM6 · 2017-12-22T01:30:13Z

src/key.cpp

-    RandAddSeedPerfmon();
    do {
-        GetRandBytes(vch, sizeof(vch));
+        GetStrongRandBytes(vch, sizeof(vch));


Could probably be a good idea to use GetStrongRandBytes in CMnemonic::Generate too

UdjinM6 · 2017-12-22T01:33:18Z

doc/benchmarking.md

+Benchmarking
+============
+
+Bitcoin Core has an internal benchmarking framework, with benchmarks


This file should be dashified

UdjinM6 · 2017-12-22T01:34:09Z

doc/benchmarking.md

+============
+
+Bitcoin Core has an internal benchmarking framework, with benchmarks
+for cryptographic algorithms such as SHA1, SHA256, SHA512 and RIPEMD160. As well as the rolling bloom filter.


Side note/TODO: maybe we should add a benchmark for X11 as well (in a separate PR)

…view

…pired dddd04f [qa] Refactor RPCTestHandler to prevent TimeoutExpired (MarcoFalke)

codablock · 2017-12-22T05:38:45Z

Pushed fixes.

Build is still failing for various reasons. One was a timeout due to stdout/stderr buffers running full, added Bitcoin bitcoin#8750 to fix this. Awaiting build result.

Another reason is that for some reason redirect_stderr is sometimes completely ignored, making the tests still fail...investigating.

I created 2 issues to not forget about the TODOs:
#1805
#1806

For some reason, subprocess.STDOUT is completely ignored when run on Travis. Using sys.stdout however works as expected.

UdjinM6

Just one small issue with dashifying otherwise 👍

UdjinM6 · 2017-12-22T13:18:45Z

doc/benchmarking.md

+for cryptographic algorithms such as SHA1, SHA256, SHA512 and RIPEMD160. As well as the rolling bloom filter.
+
+After compiling dash-core, the benchmarks can be run with:
+`src/bench/bench_bitcoin`


Missed one, should be src/bench/bench_dash

UdjinM6 · 2017-12-22T14:20:30Z

Slightly tested, ACK

laanwj and others added 30 commits December 21, 2017 17:30

Merge bitcoin#7953: Create signmessagewithprivkey rpc

90b00cf

7db0ecb Test for signing messages (Andrew Chow) f90efbf Create signmessagewithprivkey rpc (Andrew)

Merge bitcoin#8011: don't run ThreadMessageHandler at lowered priority

c66fcf4

e53e7c5 don't run ThreadMessageHandler at lowered priority (Kaz Wesley)

Merge bitcoin#7814: [qa] Switch to py3

25147c9

fa389d4 [qa] Switch to py3 (MarcoFalke)

Merge bitcoin#8013: doc: Fedora build requirements, add gcc-c++ and f…

19af9cf

…ix typo b3d18ba doc: Fedora build requirements, add gcc-c++ and fix typo (Warren Togami)

Merge bitcoin#8009: Docs: Fixed invalid example paths in gitian-build…

65a71dc

…ing.md b06f6a9 Fixed invalid example paths in gitian-building.md (JeremyRand)

Merge bitcoin#7934: Improve rolling bloom filter performance and benc…

93be53e

…hmark 1953c40 More efficient bitsliced rolling Bloom filter (Pieter Wuille) aa62b68 Benchmark rolling bloom filter (Pieter Wuille)

Merge bitcoin#7976: Remove obsolete reference to CValidationState fro…

5222e93

…m UpdateCoins. c8b9248 Remove obsolete reference to CValidationState from UpdateCoins. (21E14)

Use fThread = true default argument in CAlert::Notify

3d665aa

Instead of fully backporting Bitcoin bitcoin#7958, we make true the default value for fThread. Otherwise the AlertNotify test case will fail as it expects the alerts to be processed synchronously.

Merge bitcoin#8030: test: Revert fatal-ness of missing python-zmq

19a6c77

65fee8e test: Revert fatal-ness of missing python-zmq (Wladimir J. van der Laan)

Merge bitcoin#8018: Autofind rpc tests --srcdir

24ff341

5ea4508 Autofind rpc tests --srcdir (Jonas Schnelli)

Merge bitcoin#8012: Qt: Delay user confirmation of send

262e857

3902a29 Qt: Delay user confirmation of send (Tyler Hardin)

Merge bitcoin#8019: Remove state arg from ReconsiderBlock, rename to …

3dbae2d

…ResetBlockFailureFlags 657e07e Rename ReconsiderBlock func to reflect real behavior (instagibbs) addb9d2 Remove state arg from ReconsiderBlock (instagibbs)

Merge bitcoin#8036: init: Move berkeleydb version reporting to wallet

4750be1

3e2c946 init: Move berkeleydb version reporting to wallet (Wladimir J. van der Laan)

Merge bitcoin#8028: Fix insanity of CWalletDB::WriteTx and CWalletTx:…

76b72a7

…:WriteToDisk 0fd5997 Fix insanity of CWalletDB::WriteTx and CWalletTx::WriteToDisk (Patrick Strateman)

Merge bitcoin#7972: [qa] pull-tester: Run rpc test in parallel

f1996dc

ccccc59 [qa] Add option --portseed to test_framework (MarcoFalke) fa494de [qa] pull-tester: Run rpc test in parallel (MarcoFalke)

Merge bitcoin#8039: bench: Add crypto hash benchmarks

4ee3858

32114dd bench: Add crypto hash benchmarks (Wladimir J. van der Laan)

Merge bitcoin#8041: [qa] Fix bip9-softforks blockstore issue

f0fe21d

fad60b3 [qa] Fix bip9-softforks blockstore issue (MarcoFalke)

Merge bitcoin#8004: signal handling: fReopenDebugLog and fRequestShut…

26f07f8

…down should be type sig_atomic_t 3262316 fReopenDebugLog and fRequestShutdown should be type sig_atomic_t (Chirag Davé)

Merge bitcoin#8059: Remove unneeded feerate param from RelayTransacti…

7b25879

…on/AcceptToMemoryPool. d87b198 Remove unneeded feerate param from RelayTransaction/AcceptToMemoryPool. (Gregory Maxwell)

Merge bitcoin#8031: improvement to readability

72bee9e

fe80102 changing "(tests are) automatically run" to correspond to the earlier instance of "run automatically (on the build server)" (Matthew English)

Merge bitcoin#7932: CAddrMan::Deserialize handle corrupt serializatio…

0450dfd

…ns better. fb26bf0 CAddrMan::Deserialize handle corrupt serializations better. (Patrick Strateman)

Fix typo adddrman to addrman as requested in bitcoin#8070

8eeaa4f

Merge bitcoin#8056: [qa] Remove hardcoded "4 nodes" from test_framework

bcecdcd

fad68f7 [qa] Reduce node count for some tests (MarcoFalke) fac9349 [qa] Remove hardcoded "4 nodes" from test_framework (MarcoFalke)

laanwj and others added 7 commits December 21, 2017 18:50

Merge bitcoin#8115: Avoid integer division in the benchmark inner-mos…

a3c6303

…t loop. 63ff57d Avoid integer division in the benchmark inner-most loop. (Gregory Maxwell)

Merge bitcoin#8110: [Doc] Add benchmarking notes

6dc4a13

1a8c4d5 [Doc] Add benchmarking notes (fanquake)

Cast command to str when calling show_debug_msg

2924b34

Throws exception otherwise.

Python 3 fixes/porting in rpc tests

69dd8f4

Pass -createwalletbackups=0 to dashd when running rpc tests

fbe36d3

It was always complaining about failed backups due to dashd restarting too fast. Additionally, since the recent RPC test framework backports, all tests are expected to print NOTHING to stderr, otherwise they fail.

Introduce redirect_stderr argument to start_node

21aaf92

Also call it with redirect_stderr=True in all cases were non-critical warnings are expected.

Dashify addresses/keys in signmessages.py RPC test

a1111cd

UdjinM6 requested changes Dec 22, 2017

View reviewed changes

Python 3 in .travis.yml

249167a

codablock force-pushed the pr_backport_bitcoin_0.13-4 branch from b46920e to 249167a Compare December 22, 2017 04:40

codablock mentioned this pull request Dec 22, 2017

Implement test case for autobackup warnings "Wasn't able to create wallet backup folder" #1805

Open

codablock and others added 5 commits December 22, 2017 06:00

Use GetStrongRandBytes in CMnemonic::Generate

abe84cb

Align sorting in OverviewPage::SetupTransactionList with transaction …

b1df5e9

…view

Revert "ReprocessBlocks"->"ResetBlockFailureFlags" in debug message

5ba21d4

Dashify a few docs and the use of an address

5496e46

Merge bitcoin#8750: [qa] Refactor RPCTestHandler to prevent TimeoutEx…

a0481f1

…pired dddd04f [qa] Refactor RPCTestHandler to prevent TimeoutExpired (MarcoFalke)

codablock force-pushed the pr_backport_bitcoin_0.13-4 branch 4 times, most recently from a684455 to ef90ac1 Compare December 22, 2017 07:19

Pass sys.stdout to Popen instead of subprocess.STDOUT

475dbd4

For some reason, subprocess.STDOUT is completely ignored when run on Travis. Using sys.stdout however works as expected.

codablock force-pushed the pr_backport_bitcoin_0.13-4 branch from 8e23840 to 475dbd4 Compare December 22, 2017 07:52

UdjinM6 requested changes Dec 22, 2017

View reviewed changes

Dashify benchmarking.md binary name

34b7fdd

UdjinM6 approved these changes Dec 22, 2017

View reviewed changes

UdjinM6 added this to the 12.3 milestone Dec 22, 2017

UdjinM6 merged commit af1233c into dashpay:develop Dec 22, 2017

codablock deleted the pr_backport_bitcoin_0.13-4 branch December 22, 2017 14:43

Backport missing PRs from Bitcoin 0.13 - Part 4 #1804

Backport missing PRs from Bitcoin 0.13 - Part 4 #1804

Uh oh!

Conversation

codablock commented Dec 21, 2017

Uh oh!

UdjinM6 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

UdjinM6 Dec 22, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codablock commented Dec 22, 2017

Uh oh!

UdjinM6 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

UdjinM6 commented Dec 22, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

UdjinM6 left a comment •

edited

Loading

UdjinM6 Dec 22, 2017 •

edited

Loading