Unexpected results from the /tweaks endpoint

[cygnet3]

I'm currently building blindbit-oracle from commit 608321534e374930bee1338c298748e931289e00.
I'm testing out the /tweaks endpoint, but I'm not understanding the result I'm getting.

I'm inspecting block 214953 on signet. This block contains 8 sp-eligible transactions.

Looking at the /tweaks endpoint (dust limit + cut-through):

curl --request GET  'http://localhost:8000/tweaks/214953' | jq 'length'
8

One would expect this endpoint to return an empty array, since every output in this block has been spent. But instead, we get all 8 tweaks.

When adding a dustLimit parameter:

curl 'http://localhost:8000/tweaks/214953?dustLimit=80000' | jq 'length'
8


curl 'http://localhost:8000/tweaks/214953?dustLimit=85000' | jq 'length'
4

Here we see that 4 transactions are being filtered out with this dust limit. Looking at all txs, 4 transactions contain an output that has <85000 sats, so these are likely the ones being filtered out. These txs also contain other (spent) outputs, but I suppose these are ignored (since this is the endpoint with cut-through).

To me, these results don't make a lot of sense. I have noticed that on Signet, a lot of addresses are being reused. Could it be that the addresses being reused affects the tweaks index?

Here's my blindbit.toml for reference:

blindbit.toml

host = "127.0.0.1:8000"
chain = "signet"
rpc_endpoint = "http://127.0.0.1:38332"
rpc_pass = "..."
rpc_user = "..."
sync_start_height = 200000
max_parallel_tweak_computations = 4
max_parallel_requests = 4
tweaks_only = 0
tweaks_full_basic = 0
tweaks_full_with_dust_filter = 1
tweaks_cut_through_with_dust_filter = 1

[setavenger]

Hi,
thanks for opening an issue.

Regarding the first part, without cut-through those tweaks seem correct to me. I assume the two transactions which you are referring to are 3e57c070d284f52819ae2445def378317f7faaea69d23ae5b0007491c2820557 and 4befc3eb5ed53cd6c5132c426b9921fc4bc302267db53db365d9aae0505f0376. They both don't have any taproot outputs and only 1 p2wpkh output. Oracle does not compute tweaks for those transactions.

Regarding cut-through, I agree. If oracle has fully indexed (or actually, has indexed past the block where those outputs were spent) those should have been eliminated from the outputs. I will have to look into that. I thought I had implemented that, there might be a bug preventing the proper deletion of tweaks and UTXOs.

Something that would help me debug this a bit quicker would be to know whether the utxos endpoint returns unpruned data as well. Oracle flags spent utxos and should prune all utxos for a transaction if they are all spent.

My first instinct is that address reuse should not affect this as basically all internal workings are based on txid:vout. I suspect some issues with the trigger or execution of tweak deletions after utxos have been marked as spent.

[cygnet3]

Regarding the first part, without cut-through those tweaks seem correct to me.

Yes you're completely right, should've double checked that. I removed the first section from the description.

I'll check the utxos endpoint later when I get the time.

[setavenger]

Can you try running from this commit ba5352b on dev branch? The issue was that a wrong ordering of deletions led to tweaks not properly triggering for deletion.

[cygnet3]

Just quickly going over some blocks manually, the /utxos endpoint seems to be doing the pruning correctly. Running commit ba5352b, it looks like the pruning is now done correctly for the /tweaks endpoint too!

[setavenger]

Great, are there any other problems, otherwise I will close this issue after merging to master.
Thanks again for raising this bug!