Match rejected transactions correctly by TXID or WTXID

[conradoplg]

Motivation

When receiving a mempool-bound transaction we check if it's in the rejected list, and if it is, we avoid downloading/verifying it.

However, this check currently uses UnminedTxId which is the WTXID for V5-onward.

Specifications

https://zips.z.cash/zip-0401

Each node also MUST hold a FIFO queue RecentlyEvicted of pairs (txid, time), where the time indicates when the transaction with the given txid was evicted. The txid (rather than the wtxid ...) is used even for version 5 transactions after activation of NU5.

Designs

For rejected invalid transactions (i.e. transactions that failed verification or storage), Zebra should match:

• transactions with failed effecting data (SpendConflict, Expired, missing inputs, nullifier conflicts): check for the same TXID (mined_id()) - this ticket
• other rejection reasons: check for the same WTXID (UnminedTxId) - Add transactions that failed verification to the mempool rejected list #2818

If it's difficult to distinguish between verifier errors, all verification failures can be matched by WTXID. Then we can open a follow-up ticket to match missing inputs, nullifier conflicts, and Expired by mined_id().

This may require some refactoring to allow looking up rejected transactions by their TXID, or splitting the rejected list in two (evicted and rejected?)

Related Work

Invalid transactions are not currently being added to the rejected list; this is #2818, so that must be done before this one.

[daira]

Check if there are any rejected invalid transactions (i.e. transactions that failed verification) with the same WTXID (UnminedTxId).

If the reason for failed verification was a problem with the witness data (i.e. the transaction is otherwise valid), then if it's submitted with different witness data you need to revalidate that. That is, you probably need caches for which txids refer to transactions that are invalidated regardless of witness data, and which wtxids are invalidated by the witness data.

Apologies for the lack of explicit specification here. I'm not entirely sure zcashd does it right; it's something we should check.

[str4d]

zcashd currently has the same split that this issue was opened to create:

• A Bloom filter that tracks wtxids for invalid transactions.
• A list that tracks txids for evicted transactions.

We could add another Bloom filter to track txids of transactions that are rejected due to invalid effecting data, but the zcashd consensus stack does not currently make that distinction. I'd also argue it would only provide a modest performance optimisation, as I believe the most costly part of transaction validation is verifying the authorizing data (proofs and signatures), which a txid filter would not prevent.

[teor2345]

I unblocked this task by moving the blocked parts into other tickets (ZIP-401 and #2818).