Upgrade transfers_enrich macro and benchmark prices.hour

[Hosuke]

Benchmark macro's performance with prices.usd and prices.hour

[Hosuke]

A CI error I am not very sure here:
https://github.com/duneanalytics/spellbook/actions/runs/11614432989/job/32342585374?pr=7077#step:12:33
cc. @jeff-dude

[Hosuke]

Test result using 10 days:
Coverage comparison: https://dune.com/queries/4228497

table_name	total_rows	rows_with_amount_usd	percentage_with_amount_usd
Prices.hour Table	217822608	1520	0.0006978155362091708
Prices.usd Table	217822608	1717808	0.7886270464634231

Different rows:
https://dune.com/queries/4227035

[jeff-dude]

Test result using 10 days: Coverage comparison: https://dune.com/queries/4228497

table_name total_rows rows_with_amount_usd percentage_with_amount_usd
Prices.hour Table 217822608 1520 0.0006978155362091708
Prices.usd Table 217822608 1717808 0.7886270464634231
Different rows: https://dune.com/queries/4227035

this difference in coverage is shocking. the new dataset is supposed to have 100x or more coverage of tokens. is this isolated to blast chain, where the new prices tables haves v few tokens? or is it possible the queries have a bug?

[jeff-dude]

A CI error I am not very sure here: https://github.com/duneanalytics/spellbook/actions/runs/11614432989/job/32342585374?pr=7077#step:12:33 cc. @jeff-dude

not sure on this CI error. is this happening every run? i guess not, since you were able to build test tables?

i am able to query both the view prices.hour and raw table the view is built on dune.dune.prices_hour_raw on the app just fine

[Hosuke]

Test result using 10 days: Coverage comparison: https://dune.com/queries/4228497
table_name total_rows rows_with_amount_usd percentage_with_amount_usd
Prices.hour Table 217822608 1520 0.0006978155362091708
Prices.usd Table 217822608 1717808 0.7886270464634231
Different rows: https://dune.com/queries/4227035

this difference in coverage is shocking. the new dataset is supposed to have 100x or more coverage of tokens. is this isolated to blast chain, where the new prices tables haves v few tokens? or is it possible the queries have a bug?

Though prices.hour has more tokens in the list than prices.usd:
https://dune.com/queries/4232153

prices.usd covers far more transfers than prices.hour(not sure why):
https://dune.com/queries/4232188

Here is an overview of overlapping tokens count:
https://dune.com/queries/4232136

I think it is possible my queries have a bug...
(We may need the gap tests merged first.)

Another possibility is the prices.hour is not ready for the last few days, I should try with another time period.

[jeff-dude]

Test result using 10 days: Coverage comparison: https://dune.com/queries/4228497
table_name total_rows rows_with_amount_usd percentage_with_amount_usd
Prices.hour Table 217822608 1520 0.0006978155362091708
Prices.usd Table 217822608 1717808 0.7886270464634231
Different rows: https://dune.com/queries/4227035

this difference in coverage is shocking. the new dataset is supposed to have 100x or more coverage of tokens. is this isolated to blast chain, where the new prices tables haves v few tokens? or is it possible the queries have a bug?

Though prices.hour has more tokens in the list than prices.usd: https://dune.com/queries/4232153

prices.usd covers far more transfers than prices.hour(not sure why): https://dune.com/queries/4232188

Here is an overview of overlapping tokens count: https://dune.com/queries/4232136

I think it is possible my queries have a bug... (We may need the gap tests merged first.)

Another possibility is the prices.hour is not ready for the last few days, I should try with another time period.

yes, use random historical timeframe to test to be safe. something like oct 1st 2024 - oct 10th 2024 (or something like that)

[Hosuke]

Adding transfers_end_date param for benchmark and testing.

[Hosuke]

ethereum.transfers coverage test:
https://dune.com/queries/4259419

table_name	total_rows	rows_with_amount_usd	percentage_with_amount_usd
Prices.hour Table	5162151	1578406	30.576517424616213
Prices.usd Table	5162151	2193279	42.48769553622124

blast.transfers coverage test:
https://dune.com/queries/4259432

table_name	total_rows	rows_with_amount_usd	percentage_with_amount_usd
Prices.hour Table	4282513	1281094	29.91453849643889
Prices.usd Table	4282513	1684401	39.332069745030545

[Hosuke]

Repeat the experiments above, a slightly different result:

ethereum.transfers coverage test:
https://dune.com/queries/4331351

table_name	total_rows	rows_with_amount_usd	percentage_with_amount_usd
Prices.hour Table	5655920	2076310	36.710384871073146
Prices.usd Table	5162151	2193965	42.50098457019177

blast.transfers coverage test:
https://dune.com/queries/4331536

table_name	total_rows	rows_with_amount_usd	percentage_with_amount_usd
Prices.hour Table	4687341	1689902	36.05246556629867
Prices.usd Table	4282513	1684401	39.332069745030545