jmcorgan address index not updated immediately upon tx broadcast

[adamkrellenstein]

Reported by Bter.

[cryptonaut420]

I believe this is happening to me as well. when sending several transactions at once with unconfirmed outputs enabled, it will quickly run out of unspent inputs, and then all transactions after the first couple fail (bitcoind error -25.. no error message). Running the batch of transactions again a minute or so later, a couple more will succeed and then the rest again fail. Previously, it would just chain together the outputs of all the transactions and all would produce successful tx ids

for example, when using this script: https://github.com/tokenly/tokenly-cms/blob/master/scripts/xcp-sendmany.py

[btcdrak]

Ping @reorder

[reorder]

@btcdrak This is not relevant to CH as of yet, as we combine addrindex transactions with mempool in addrindex driver.

[adamkrellenstein]

@reorder, can you elaborate on that?

[btcdrak]

@adamkrellenstein This is what I was telling about previously as the patch is vastly superior to what was proposed.

[adamkrellenstein]

@btcdrak, I don't understand. The addrindex patch you applied to viacoin doesn't have this problem?

[reorder]

@adamkrellenstein No, rather our driver accessing viacoin merges searchrawtransactions output with mempool.

[btcdrak]

@adamkrellenstein Additionally for reference, we now have a patch now that will apply to Bitcoin Core 0.10 branch (which will be released soon as 0.10.0) it will cherry-pick to Bitcoin. I've published a branch here.

[adamkrellenstein]

Got it. @ouziel-slama, what do you think?

[ouziel-slama]

In counterpartyd we are doing the same thing as viacoin:

viacoin merges searchrawtransactions output with mempool

here: https://github.com/CounterpartyXCP/counterpartyd/blob/master/lib/blockchain/jmcorgan.py#L15

About the error -25, since forever Bitcoin Core (not in particularly jmcorcan branch) return it when you try to chain unconfirmed transactions too fast. For instance, it was already occurred when we developed the sweeping system in the Counterwallet. Maybe jmcorgan branch is more "sensitive", but even with the master branch, there is no choice except to wait few seconds and retry again (is what we are doing for the sweeping).

[btcdrak]

@ouziel-slama Our driver is much more complex see mempoolsearch and this

[cryptonaut420]

@ouziel-slama for the error -25, yeah I have noticed that happening before with making around 30+ transactions, but with this patch that number has gone down to like around 5 or 6 at most. Looking at the payloads it is sending in the failed transactions, it looks like its trying to re-use unspent transactions which have already been used, causing them to fail.

[ouziel-slama]

@btcdrak, nop! see the 4 functions following this line:
https://github.com/CounterpartyXCP/counterpartyd/blob/master/lib/util.py#L1111
To avoid a loop with a lot of RPC calls (like here), we have a "complex" cache system. If we are not using this cache, we are getting very quickly timeout errors in the Counterwallet.

After thoughts, maybe the last version is more sensitive not because jmcorgan but because our cache system. I think it's because this line: https://github.com/CounterpartyXCP/counterpartyd/blob/master/lib/blocks.py#L1614.

@adamkrellenstein, what do you think to add in the config file a variable USE_MEMPOOL_CACHE ?

[adamkrellenstein]

@ouziel-slama, why would that mempool line matter here? How would USE_MEMPOOL_CACHE work?

Is the delay because of https://github.com/CounterpartyXCP/counterpartyd/blob/master/lib/blocks.py#L1686 and https://github.com/CounterpartyXCP/counterpartyd/blob/master/lib/blocks.py#L1326?

[ouziel-slama]

why would that mempool line matter here? How would USE_MEMPOOL_CACHE work?

@adamkrellenstein, this line matter to avoid as much as possible RPC calls.
if USE_MEMPOOL_CACHE is False counterpartyd would be using a simple loop with a lot of RPC calls (like here) else it would using the same system as now, and would be more "sensitive" to the error -25.

Is the delay because of ...

yes, and also by the time to parse transactions.

[reorder]

Perhaps a correct way to solve this dilemma is to communicate with bitcoind over p2p, refreshing mempool cache on 'inv' messages.

[adamkrellenstein]

Fixed: #525