Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ETH Transfers to L1 #2440

Merged
merged 8 commits into from
Jan 9, 2023
Merged

Conversation

MSilb7
Copy link
Contributor

@MSilb7 MSilb7 commented Jan 6, 2023

Brief comments on the purpose of your changes:

For Dune Engine V2

Add ETH transfers to L1 & make small type fixes to ETH transfers on OP.

The /eth/ subfolder in Optimism was removed in #2426 so keeping that consistent here.

I've checked that:

General checks:

  • I tested the query on dune.com after compiling the model with dbt compile (compiled queries are written to the target directory)
  • I used "refs" to reference other models in this repo and "sources" to reference raw or decoded tables
  • if adding a new model, I added a test
  • the filename is unique and ends with .sql
  • each sql file is a select statement and has only one view, table or function defined
  • column names are lowercase_snake_cased
  • if adding a new model, I edited the dbt project YAML file with new directory path for both models and seeds (if applicable)
  • if wanting to expose a model in the UI (Dune data explorer), I added a post-hook in the JINJA config to add metadata (blockchains, sector/project, name and contributor Dune usernames)

Pricing checks:

  • coin_id represents the ID of the coin on coinpaprika.com
  • all the coins are active on coinpaprika.com (please remove inactive ones)

Join logic:

  • if joining to base table (i.e. ethereum transactions or traces), I looked to make it an inner join if possible

Incremental logic:

  • I used is_incremental & not is_incremental jinja block filters on both base tables and decoded tables
    • where block_time >= date_trunc("day", now() - interval '1 week')
  • if joining to base table (i.e. ethereum transactions or traces), I applied join condition where block_time >= date_trunc("day", now() - interval '1 week')
  • if joining to prices view, I applied join condition where minute >= date_trunc("day", now() - interval '1 week')

@dune-eng
Copy link

dune-eng commented Jan 6, 2023

Workflow run id 3855823700 approved.

@dune-eng
Copy link

dune-eng commented Jan 6, 2023

Workflow run id 3855823701 approved.

@dune-eng
Copy link

dune-eng commented Jan 6, 2023

Workflow run id 3855828016 approved.

@dune-eng
Copy link

dune-eng commented Jan 6, 2023

Workflow run id 3855828011 approved.

@dune-eng
Copy link

dune-eng commented Jan 6, 2023

Workflow run id 3858079054 approved.

@dune-eng
Copy link

dune-eng commented Jan 6, 2023

Workflow run id 3858079071 approved.

@dune-eng
Copy link

dune-eng commented Jan 6, 2023

Workflow run id 3858089894 approved.

@dune-eng
Copy link

dune-eng commented Jan 6, 2023

Workflow run id 3858089891 approved.

@dune-eng
Copy link

dune-eng commented Jan 7, 2023

Workflow run id 3862278357 approved.

@dune-eng
Copy link

dune-eng commented Jan 7, 2023

Workflow run id 3862278358 approved.

@dune-eng
Copy link

dune-eng commented Jan 7, 2023

Workflow run id 3863376505 approved.

@dune-eng
Copy link

dune-eng commented Jan 7, 2023

Workflow run id 3863376506 approved.

@soispoke soispoke self-assigned this Jan 9, 2023
@soispoke
Copy link
Contributor

soispoke commented Jan 9, 2023

lgtm, thanks @MSilb7

@soispoke soispoke self-requested a review January 9, 2023 13:40
@soispoke soispoke merged commit d5533bf into duneanalytics:main Jan 9, 2023
@gitpoap-bot
Copy link

gitpoap-bot bot commented Jan 9, 2023

Congrats, your important contribution to this open-source project has earned you a GitPOAP!

GitPOAP: 2023 Dune Contributor:

GitPOAP: 2023 Dune Contributor GitPOAP Badge

Head to gitpoap.io & connect your GitHub account to mint!

Learn more about GitPOAPs here.

jeff-dude pushed a commit to MSilb7/spellbook that referenced this pull request Jan 9, 2023
* add eth transfers and small type fixes in op eth transfers

* comment update

* update schema

* update transfer id

* add tx to and from

* add tx_to and tx_from

* nit remove spaces

Co-authored-by: soispoke <66172107+soispoke@users.noreply.github.com>
jeff-dude added a commit that referenced this pull request Jan 10, 2023
* fix perp contract

* update token generations & move files

* mod aave sources

* update erc20 schema

* mod atoken sources

* fix indent

* aave sources alignment

* update aave schema again

* granary sources update

* more schema updates

* update ovm_optimism schema

* mod aave schema

* whoops - fix commas

* fix commas

* take - out of descriptions

* fixes

* expose OP erc20s as a separate detailed list

* map op back to erc20 tokens

* add standard to nft for op

* update dbt project

* add sided

* Update contracts_optimism_contract_creator_address_list.sql

* small

* typo

* Update tokens_optimism_schema.yml

* adds

* add dependency on snapshot

* Fix dbt_project.yml

* Update tokens_optimism_nft.sql

* remove generated tokens for now

* initialize without circular dependencies

* update prices

* update btc.b

* forget snapshot

* remove deps

* aave chain insert

* fix aave for loop

* fix atokens select

* simplify

* alias

* fix gran

* fix gran

note - granary initialized events are missing

* comment out descriptor

* erc20 schema

* fix

* Update tokens_optimism_erc20.sql

* price dupe

* update lists

* nft list

* fix unique nft

* add attestation override

* add mm swaps

* Update tokens_optimism_nft.sql

* add gysr

* add nft

* add erc20

* quix <> rh contract

* stick

* add type on atokens

* add woofi

* adds

* Reformat aave_v3_tokens for compatibility

* Fix typo

* Add uniqueness test

* Remove schema in config block

* Reformat granary_tokens and compatibility for Trino

* add token to prices

* adds

* add creator and update nft name

* revert OaycNFT renaming

* Apply CAST() to number_of_items and amount_raw

* fix tofu_bnb schema

* Add CAST() to fields

* fix fee convention

* Remove hardcoded fields from nft_trades

* convert to hours (#2399)

* [Easy] [Label] Implement trader platforms label (#2394)

* Implemented trader platforms label

* Fixed errors in spell

* Added forgotten group by

* Fixed columns selection

* Added ordering and keeping of order for label

* Fixed error in brackets

* Added missing block_time column

* Added time to keep order

* [easy] new tokens (#2414)

* Adding Aave V3 assets to tokens_avalanche_c_erc20.sql and tokens_polygon_erc20.sql (#2410)

* add AAVE, EURS to arbitrum_erc20.sql

* adding remaining aave v3 assets to erc20

* [CoW Protocol] Add Referral Mapping (#2377)

* add referral table

* remove duplicate anchor

* remove unused subquery

* Add Pancakeswap NFT Events (#2255)

* add seed in yml file

* Add Pancakeswap NFT V1 to nft trades

* delete tofu

* Adjust

* update

* update LOGIC

* initial commit

* update

* Update models/pancakeswap/bnb/pancakeswap_bnb_nft_market_v1_event.sql

Co-authored-by: 0xRob <83790096+0xRobin@users.noreply.github.com>

* Update models/pancakeswap/bnb/pancakeswap_bnb_nft_market_v1_event.sql

Co-authored-by: 0xRob <83790096+0xRobin@users.noreply.github.com>

* add test

* add

* Update models/pancakeswap/bnb/pancakeswap_bnb_nft_market_v1_event.sql

Co-authored-by: 0xRob <83790096+0xRobin@users.noreply.github.com>

* delete liquidifty

* add test file

* .

* update

* fix unique ID

* update sample file

* change pancakeswap seed schema

* temporary comment trade section

* reinstate test and correct columns

* change sample file

* change seed file

* add pancakeswap to nft trades

* add royalty info

* fix currency_symbol

* guard fee logic

* cast null as string

* remove double column

Co-authored-by: 0xRob <83790096+0xRobin@users.noreply.github.com>
Co-authored-by: 0xRob <0xRobin5@gmail.com>

* Migrate Balancer `vebal_slopes` abstraction (#2179)

* Add balancer_vebal_slopes to schema yml file

* Add veBAL related sources to sources yml file

* Add balancer_vebal_slopes spell

* Reorder sources

* Change casting syntax

* Rename datetime field from schema .yml file

* Add incremental logic to vebal_slopes spell

* Update models/balancer/ethereum/balancer_vebal_slopes.sql

Co-authored-by: Huang Geyang <Sukebeta@outlook.com>

* Update models/balancer/ethereum/balancer_vebal_slopes.sql

Co-authored-by: Huang Geyang <Sukebeta@outlook.com>

* Update models/balancer/ethereum/balancer_ethereum_schema.yml

Co-authored-by: Huang Geyang <Sukebeta@outlook.com>

* Update models/balancer/ethereum/balancer_vebal_slopes.sql

Co-authored-by: Huang Geyang <Sukebeta@outlook.com>

* Replace block_time unique key for block_number

* Update models/balancer/ethereum/balancer_ethereum_schema.yml

Co-authored-by: Huang Geyang <Sukebeta@outlook.com>

* Update models/balancer/ethereum/balancer_vebal_slopes.sql

Co-authored-by: Huang Geyang <Sukebeta@outlook.com>

* Replace block_date for block_number in uniqueness test

* Rewrite group by in full column names

* Fix unmatched columns in schema.yml

* add descriptions to vebal_slopes columns

Co-authored-by: Huang Geyang <Sukebeta@outlook.com>
Co-authored-by: Fábio Mendes <ms.fabiomendes@gmail.com>

* Update cow_protocol_ethereum_referrals.sql (#2423)

* Fix Source Chain ID/Name Hop Optimism (#2313)

* Update hop_protocol_flows.sql

* Revert "Update hop_protocol_flows.sql"

This reverts commit be30148.

* Update hop_protocol_optimism_flows.sql

* Add BSC Dodo trades to dex.trades (#2157)

* dodo-bnb-trades-real

* add seed tests

* seed value typo

* remove depracated test

* Modify dodo_bnb to aligh with dodo_eth in project.yml

* combine cross-chain bnb & ethereum trades

* correct ethereum dodo trades

* downgrade 'dbt_utils'

* Change back dbt_utils version to 0.9.2

* final tweaks

* force failure

* Remove extra line

* correct seed file

* restore deleted prices

* minor formatting

* cleanup dbt_project with latest standard

* add freshness check

Co-authored-by: Huang Geyang <Sukebeta@outlook.com>
Co-authored-by: jeff-dude <jeff@dune.xyz>

* script to find sources missing checks (#2401)

* script to find sources missing checks + new freshness checks

* T-22136: Missing ERC20 tokens (#2425)

* add A LOT of missing tokens

* sort + indentation

* escape backtick

* add missing comma

* remove last comma

* remove duplicate

* add trim to symbol

* remove trim, see if test pass

* fixed trim

Co-authored-by: jeff-dude <jeff@dune.xyz>

* Update pull_request_template.md (#2429)

* Update README.md (#2411)

* T-23065 - optimism transfers: update value filter for string datatype (#2426)

* update value filter for string datatype

* directory cleanup

* Seaport ethereum patch (#2419)

* unifying seaport source

* keep unique_trade_id for opensea.trades for nft.trades compatability

* merge schema file

* cast amount to decimal(38,0)

* add test seeds

* update schema.yml

* Fix: minor typos (#2424)

* Fix: minor typos

Fix: minor typos

* Fix: minor typo

Fix: minor typo

* Fix: minor typo

Fix: minor typo

* Fix: minor typo

Fix: minor typo

* [Quick] Updates to metadata files (#2421)

* add new metadata

* fix duplicate error

Co-authored-by: cat <cat377550@gmail.com>

* Added alphasharks.io as an aggregator (#2417)

* Fix aave liquidation call (#2432)

* FIX: cast to varchar in opensea_v3 trades (#2433)

* cast to varchar in concat in view

* define varchar length'

* use batch api calls for price checker (#2431)

* use batch api calls for price checker

* Update dodo_ethereum_trades & add dodo_bnb_trades & split them to aggregator and pools (#2169)

* update dodo_ethereum_trades && add dodo_bnb_trades

* split dodo_trades to pools & aggregator on Ethereum and BNBChain

* label different dodoV2 pools

* fix schema & sql

* add dodo_proxies on Ethereum

* Modify dodo_bnb_trades to incremental view

* Apply check_dex_seed and modify dodo_eth_trades to incremental view

* Add DODO bnb trade seeds

* Apply check_dex_seed to DODO bnb trades model

* update dex_trade_seeds

* fix

* Add DODO bnb in project.yml

* remove gitkeep in dodo

* Update models/dodo/bnb/dodo_pools_bnb_trades.sql

* Update check_dex_seed in dodo_ethereum_schema.yml

* fix dodo_ethereum_schema

* Remove schema in the config blocks

* Add dodo_trades files by scoffie

* add dppOracle & dppAdvanced to dodo_bnb

* fix

* update dbt_project on dodo

* fix dodo_bnb_sources.yml

* move cast to base model

* update dex.trades seeds on DODO_BNBChain

* final formatting & cleanup expose spell macro

Co-authored-by: Huang Geyang <Sukebeta@outlook.com>
Co-authored-by: jeff-dude <jeff@dune.xyz>

* [CoW] Fix Fee USD Calculation (#2422)

* use most natural form of fee computation when possible

* add comment about imprecision

* Fix failing tests (#2396)

* trigger test

* enable test in CI

* defer in test

* fix aragon

* fix native token test

* fix native mints unique id

* fix nft trader labels

* union all in nft labels

* union all for performance

* trigger wash_trades full refresh

* fix cow protocol distinct labels

* revert null wallet_address fix

Co-authored-by: soispoke <66172107+soispoke@users.noreply.github.com>

* Add ETH Transfers to L1 (#2440)

* add eth transfers and small type fixes in op eth transfers

* comment update

* update schema

* update transfer id

* add tx to and from

* add tx_to and tx_from

* nit remove spaces

Co-authored-by: soispoke <66172107+soispoke@users.noreply.github.com>

* Token checker tweaks (#2437)

* allow for case insensitive token symbol matching

* adapt token checker error/warns

* check for warning in test instead of error

* Allow for local compile of incremental models.  (#2403)

* base is_incremental

* add force-incremental variable

* clean up dbt_project values

* add version schema name

* rename ovm directory

* naming standards

* keep optimism erc20 view name

Co-authored-by: Huang Geyang <Sukebeta@outlook.com>
Co-authored-by: 0xRob <0xRobin5@gmail.com>
Co-authored-by: Belén Pruvost <belen@dune.com>
Co-authored-by: Meghan Heintz <meghan@dune.com>
Co-authored-by: Gent Rexha <gent@cow.fi>
Co-authored-by: josojo <josojo@hotmail.de>
Co-authored-by: mc <mattcole1093@gmail.com>
Co-authored-by: Benjamin Smith <bh2smith@users.noreply.github.com>
Co-authored-by: Harry <81097471+thb030100@users.noreply.github.com>
Co-authored-by: 0xRob <83790096+0xRobin@users.noreply.github.com>
Co-authored-by: Victor Stefenon <victorstefenon0@gmail.com>
Co-authored-by: Fábio Mendes <ms.fabiomendes@gmail.com>
Co-authored-by: soispoke <66172107+soispoke@users.noreply.github.com>
Co-authored-by: Levis Kithinji <64379631+LeviScoffie@users.noreply.github.com>
Co-authored-by: jeff-dude <jeff@dune.xyz>
Co-authored-by: jeff-dude <102681548+jeff-dude@users.noreply.github.com>
Co-authored-by: sohwak <59173567+sohwak@users.noreply.github.com>
Co-authored-by: omahs <73983677+omahs@users.noreply.github.com>
Co-authored-by: catherine-2 <95732099+catherine-2@users.noreply.github.com>
Co-authored-by: cat <cat377550@gmail.com>
Co-authored-by: Garv Maggu <17963209+GarvMaggu@users.noreply.github.com>
Co-authored-by: Dias <dsalv@users.noreply.github.com>
Co-authored-by: owen05 <owen05@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants