Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add zkevm to safe models #5758

Merged
merged 57 commits into from
Apr 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
5444ebf
Add v1.4.1 singletons
danielpartida Feb 20, 2024
5862466
Change goerli test
danielpartida Feb 20, 2024
761f15a
Merge branch 'main' of https://github.com/5afe/spellbook
danielpartida Apr 9, 2024
8bf1492
Declare new Safe models for zkevm
danielpartida Apr 9, 2024
39d3840
Implement Safe models for zkevm
danielpartida Apr 9, 2024
8500052
Update crosschain models with zkevm data
danielpartida Apr 9, 2024
7715348
Bugfix: Joining correctly on celo transactions for safe_celo_transact…
danielpartida Apr 9, 2024
fd5cff2
Fix project start for safe_polygon_matic_transfers.sql
danielpartida Apr 9, 2024
2c1e59e
rename correctly Safe zkevm transactions spell
danielpartida Apr 9, 2024
271afdb
Add zkevm base source
danielpartida Apr 10, 2024
c80852c
Implement correctly base source
danielpartida Apr 10, 2024
6e7558f
Fix zkevm schema
danielpartida Apr 10, 2024
20a4462
Trigger run again
danielpartida Apr 10, 2024
68d89bf
Remove singletons spell
danielpartida Apr 11, 2024
3c967d8
Add singletons spell
danielpartida Apr 11, 2024
bac8ccf
Merge branch 'main' into add-zkevm-to-safe-models
danielpartida Apr 11, 2024
ba65bab
Merge remote-tracking branch 'origin/add-zkevm-to-safe-models' into a…
danielpartida Apr 11, 2024
c93f52d
add decoded source tables
danielpartida Apr 11, 2024
29384c6
Merge branch 'main' into add-zkevm-to-safe-models
danielpartida Apr 11, 2024
42fcc98
Fix duplicate anchor
danielpartida Apr 11, 2024
02caf0f
Merge remote-tracking branch 'origin/add-zkevm-to-safe-models' into a…
danielpartida Apr 11, 2024
9f38c67
Update sources/safe/zkevm/safe_zkevm_sources.yml
danielpartida Apr 12, 2024
eef0799
Merge branch 'main' into add-zkevm-to-safe-models
danielpartida Apr 12, 2024
b233a7a
Adapt test
danielpartida Apr 12, 2024
f508ceb
Merge branch 'add-zkevm-to-safe-models' of https://github.com/5afe/sp…
danielpartida Apr 12, 2024
0480800
Merge branch 'main' into add-zkevm-to-safe-models
danielpartida Apr 15, 2024
7337493
Merge branch 'main' into add-zkevm-to-safe-models
danielpartida Apr 16, 2024
dd4a64a
Update models/safe/zkevm/safe_zkevm_matic_transfers.sql
danielpartida Apr 16, 2024
fb16441
Update models/safe/zkevm/safe_zkevm_matic_transfers.sql
danielpartida Apr 16, 2024
f963b3a
Merge branch 'main' into add-zkevm-to-safe-models
danielpartida Apr 16, 2024
7d20b93
Merge branch 'main' into add-zkevm-to-safe-models
danielpartida Apr 18, 2024
0760bbf
Add schema to zkevm Safe data models
danielpartida Apr 22, 2024
7d1bf1a
Adapt pot_hook section
danielpartida Apr 22, 2024
796ffd9
Use same incremental interval in union
danielpartida Apr 22, 2024
def7bc2
Set project start date
danielpartida Apr 22, 2024
d3506c7
Add unique combo of columns test
danielpartida Apr 22, 2024
597f98a
Merge branch 'main' into add-zkevm-to-safe-models
danielpartida Apr 22, 2024
5e44197
remove unique_key column on zkevm_transactions
danielpartida Apr 22, 2024
9c5ec38
Revert "remove unique_key column on zkevm_transactions"
danielpartida Apr 22, 2024
fa7d785
Delelete unique key on safe_zkevm_transacions
danielpartida Apr 22, 2024
eb435be
Merge branch 'add-zkevm-to-safe-models' of https://github.com/5afe/sp…
danielpartida Apr 22, 2024
0a9cdc1
Update models/safe/zkevm/safe_zkevm_matic_transfers.sql
danielpartida Apr 22, 2024
f9a367c
Add increment join
danielpartida Apr 22, 2024
d9cabc8
Address merge conflicts
danielpartida Apr 22, 2024
235cb10
Address one more merge conflict
danielpartida Apr 22, 2024
65a8e8e
Address one more merge conflict
danielpartida Apr 22, 2024
9164d33
Merge branch 'main' into add-zkevm-to-safe-models
danielpartida Apr 22, 2024
bffe5a5
Quickfix: Change block_time incremental join
danielpartida Apr 22, 2024
2f4f282
Merge remote-tracking branch 'origin/add-zkevm-to-safe-models' into a…
danielpartida Apr 22, 2024
04f7b8c
Update models/safe/zkevm/safe_zkevm_transactions.sql
danielpartida Apr 23, 2024
12fbbb7
Update models/safe/zkevm/safe_zkevm_transactions.sql
danielpartida Apr 23, 2024
4aca769
Address feedback of jeff
danielpartida Apr 23, 2024
fda8ed0
Merge branch 'main' into add-zkevm-to-safe-models
danielpartida Apr 23, 2024
b6ac672
Move unique combo test to safe txs spell
danielpartida Apr 23, 2024
f3edf5e
Merge branch 'add-zkevm-to-safe-models' of https://github.com/5afe/sp…
danielpartida Apr 23, 2024
bdafa6c
Update models/safe/zkevm/safe_zkevm_schema.yml
danielpartida Apr 24, 2024
49958d0
Merge branch 'main' into add-zkevm-to-safe-models
danielpartida Apr 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion models/safe/celo/safe_celo_transactions.sql
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ join {{ ref('safe_celo_safes') }} s
on s.address = tr."from"
join {{ ref('safe_celo_singletons') }} ss
on tr.to = ss.address
join {{ source('ethereum', 'transactions') }} et
join {{ source('celo', 'transactions') }} et
on tr.tx_hash = et.hash
and tr.block_number = et.block_number
where bytearray_substring(tr.input, 1, 4) in (
Expand Down
3 changes: 2 additions & 1 deletion models/safe/safe_native_transfers_all.sql
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{{ config(
schema = 'safe',
alias = 'native_transfers_all',
post_hook='{{ expose_spells(\'["arbitrum","avalanche_c","base","bnb","celo","ethereum","gnosis","goerli","optimism","polygon","zksync"]\',
post_hook='{{ expose_spells(\'["arbitrum","avalanche_c","base","bnb","celo","ethereum","gnosis","goerli","optimism","polygon","zkevm","zksync"]\',
"project",
"safe",
\'["kryptaki", "danielpartida"]\') }}'
Expand All @@ -19,6 +19,7 @@
,ref('safe_goerli_eth_transfers')
,ref('safe_optimism_eth_transfers')
,ref('safe_polygon_matic_transfers')
,ref('safe_zkevm_matic_transfers')
,ref('safe_zksync_eth_transfers')
] %}

Expand Down
3 changes: 2 additions & 1 deletion models/safe/safe_safes_all.sql
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{{ config(
schema = 'safe',
alias = 'safes_all',
post_hook='{{ expose_spells(\'["arbitrum","avalanche_c","base","bnb","celo","ethereum","fantom","gnosis","goerli","optimism","polygon","zksync"]\',
post_hook='{{ expose_spells(\'["arbitrum","avalanche_c","base","bnb","celo","ethereum","fantom","gnosis","goerli","optimism","polygon","zkevm","zksync"]\',
"project",
"safe",
\'["tschubotz", "danielpartida", "kryptaki"]\') }}'
Expand All @@ -20,6 +20,7 @@
,ref('safe_goerli_safes')
,ref('safe_optimism_safes')
,ref('safe_polygon_safes')
,ref('safe_zkevm_safes')
,ref('safe_zksync_safes')
] %}

Expand Down
16 changes: 8 additions & 8 deletions models/safe/safe_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ version: 2
models:
- name: safe_safes_all
meta:
blockchain: arbitrum, avalanche_c, base, bnb, celo, ethereum, fantom, gnosis, goerli, optimism, polygon, zksync
blockchain: arbitrum, avalanche_c, base, bnb, celo, ethereum, fantom, gnosis, goerli, optimism, polygon, zkevm, zksync
project: safe
contributors: tschubotz, danielpartida, kryptaki
config:
tags: ['arbitrum', 'avalanche_c', 'bnb', 'celo','ethereum', 'fantom', 'gnosis', 'goerli', 'optimism', 'polygon', 'safe','zksync']
tags: ['arbitrum', 'avalanche_c', 'bnb', 'celo','ethereum', 'fantom', 'gnosis', 'goerli', 'optimism', 'polygon', 'safe', 'zkevm', 'zksync']
description: >
Lists all Safes across chains.
columns:
Expand All @@ -32,14 +32,14 @@ models:

- name: safe_transactions_all
meta:
blockchain: arbitrum, avalanche_c, base, bnb, celo, ethereum, fantom, gnosis, goerli, optimism, polygon, zksync
blockchain: arbitrum, avalanche_c, base, bnb, celo, ethereum, fantom, gnosis, goerli, optimism, polygon, zkevm, zksync
project: safe
contributors: kryptaki
contributors: kryptaki, danielpartida
freshness:
warn_after: { count: 12, period: hour }
error_after: { count: 24, period: hour }
config:
tags: ['arbitrum', 'avalanche_c', 'bnb', 'celo','ethereum', 'fantom', 'gnosis', 'goerli', 'optimism', 'polygon', 'safe', 'zksync']
tags: ['arbitrum', 'avalanche_c', 'bnb', 'celo','ethereum', 'fantom', 'gnosis', 'goerli', 'optimism', 'polygon', 'safe', 'zkevm', 'zksync']
description: "Safe transactions"
columns:
- *blockchain
Expand Down Expand Up @@ -94,14 +94,14 @@ models:

- name: safe_native_transfers_all
meta:
blockchain: arbitrum, avalanche_c, base, bnb, celo, ethereum, gnosis, goerli, optimism, polygon, zksync
blockchain: arbitrum, avalanche_c, base, bnb, celo, ethereum, gnosis, goerli, optimism, polygon, zkevm, zksync
project: safe
contributors: tschubotz
contributors: kryptaki, tschubotz, danielpartida
freshness:
warn_after: { count: 12, period: hour }
error_after: { count: 24, period: hour }
config:
tags: ['arbitrum', 'avalanche_c', 'bnb', 'celo','ethereum', 'gnosis', 'goerli', 'optimism', 'polygon', 'safe', 'zksync']
tags: ['arbitrum', 'avalanche_c', 'bnb', 'celo','ethereum', 'gnosis', 'goerli', 'optimism', 'polygon', 'safe', 'zkevm', 'zksync']
description: "Native gas token transfers into or out of Safes"
tests:
- dbt_utils.unique_combination_of_columns:
Expand Down
3 changes: 2 additions & 1 deletion models/safe/safe_transactions_all.sql
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{{ config(
schema = 'safe',
alias = 'transactions_all',
post_hook='{{ expose_spells(\'["arbitrum","avalanche_c","base","bnb","celo","ethereum","fantom","gnosis","goerli","optimism","polygon","zksync"]\',
post_hook='{{ expose_spells(\'["arbitrum","avalanche_c","base","bnb","celo","ethereum","fantom","gnosis","goerli","optimism","polygon","zkevm","zksync"]\',
"project",
"safe",
\'["kryptaki", "danielpartida"]\') }}'
Expand All @@ -20,6 +20,7 @@
,ref('safe_goerli_transactions')
,ref('safe_optimism_transactions')
,ref('safe_polygon_transactions')
,ref('safe_zkevm_transactions')
,ref('safe_zksync_transactions')
] %}

Expand Down
82 changes: 82 additions & 0 deletions models/safe/zkevm/safe_zkevm_matic_transfers.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
{{
config(
materialized='incremental',
schema = 'safe_zkevm',
alias = 'matic_transfers',
partition_by = ['block_month'],
unique_key = ['block_date', 'address', 'tx_hash', 'trace_address'],
on_schema_change='fail',
file_format ='delta',
incremental_strategy='merge',
post_hook = '{{ expose_spells(
blockchains = \'["zkevm"]\',
spell_type = "project",
spell_name = "safe",
contributors = \'["danielpartida"]\') }}'
)
}}

{% set project_start_date = '2023-09-01' %}

select
t.*,
p.price * t.amount_raw / 1e18 AS amount_usd

from (
select
'zkevm' as blockchain,
'MATIC' as symbol,
s.address,
try_cast(date_trunc('day', et.block_time) as date) as block_date,
CAST(date_trunc('month', et.block_time) as DATE) as block_month,
et.block_time,
-CAST(et.value AS INT256) as amount_raw,
et.tx_hash,
array_join(et.trace_address, ',') as trace_address
from {{ source('zkevm', 'traces') }} et
join {{ ref('safe_zkevm_safes') }} s on et."from" = s.address
and et."from" != et.to -- exclude calls to self to guarantee unique key property
and et.success = true
and (lower(et.call_type) not in ('delegatecall', 'callcode', 'staticcall') or et.call_type is null)
and et.value > UINT256 '0' -- et.value is uint256 type
{% if not is_incremental() %}
where et.block_time > TIMESTAMP '{{project_start_date}}' -- for initial query optimisation
{% endif %}
{% if is_incremental() %}
-- to prevent potential counterfactual safe deployment issues we take a bigger interval
where et.block_time > date_trunc('day', now() - interval '10' day)
danielpartida marked this conversation as resolved.
Show resolved Hide resolved
{% endif %}

union all

select
'zkevm' as blockchain,
'MATIC' as symbol,
s.address,
try_cast(date_trunc('day', et.block_time) as date) as block_date,
CAST(date_trunc('month', et.block_time) as DATE) as block_month,
et.block_time,
CAST(et.value AS INT256) as amount_raw,
et.tx_hash,
array_join(et.trace_address, ',') as trace_address
from {{ source('zkevm', 'traces') }} et
join {{ ref('safe_zkevm_safes') }} s on et.to = s.address
and et."from" != et.to -- exclude calls to self to guarantee unique key property
and et.success = true
and (lower(et.call_type) not in ('delegatecall', 'callcode', 'staticcall') or et.call_type is null)
and et.value > UINT256 '0' -- et.value is uint256 type
{% if not is_incremental() %}
where et.block_time > TIMESTAMP '{{project_start_date}}' -- for initial query optimisation
{% endif %}
{% if is_incremental() %}
-- to prevent potential counterfactual safe deployment issues we take a bigger interval
where et.block_time > date_trunc('day', now() - interval '10' day)
danielpartida marked this conversation as resolved.
Show resolved Hide resolved
{% endif %}
) t

left join {{ source('prices', 'usd') }} p on p.blockchain is null
danielpartida marked this conversation as resolved.
Show resolved Hide resolved
{% if is_incremental() %}
and {{ incremental_predicate('p.minute') }}
{% endif %}
and p.symbol = t.symbol
and p.minute = date_trunc('minute', t.block_time)
49 changes: 49 additions & 0 deletions models/safe/zkevm/safe_zkevm_safes.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
{{
config(
materialized='incremental',
schema = 'safe_zkevm',
alias= 'safes',
partition_by = ['block_month'],
unique_key = ['block_date', 'address'],
on_schema_change='fail',
file_format ='delta',
incremental_strategy='merge',
post_hook = '{{ expose_spells(
blockchains = \'["zkevm"]\',
spell_type = "project",
spell_name = "safe",
contributors = \'["danielpartida"]\') }}'
)
}}

select
'zkevm' as blockchain,
et."from" as address,
case
when et.to = 0xd9db270c1b5e3bd161e8c8503c55ceabee709552 then '1.3.0'
when et.to = 0x3e5c63644e683549055b9be8653de26e0b4cd36e then '1.3.0L2'
when et.to = 0x41675C099F32341bf84BFc5382aF534df5C7461a then '1.4.1'
when et.to = 0x29fcB43b46531BcA003ddC8FCB67FFE91900C762 then '1.4.1L2'
else 'unknown'
end as creation_version,
try_cast(date_trunc('day', et.block_time) as date) as block_date,
CAST(date_trunc('month', et.block_time) as DATE) as block_month,
et.block_time as creation_time,
et.tx_hash
from {{ source('zkevm', 'traces') }} et
join {{ ref('safe_zkevm_singletons') }} s
on et.to = s.address
where et.success = true
and et.call_type = 'delegatecall' -- delegatecall to singleton is Safe (proxy) address
and bytearray_substring(et.input, 1, 4) in (
0x0ec78d9e, -- setup method v0.1.0
0xa97ab18a, -- setup method v1.0.0
0xb63e800d -- setup method v1.1.0, v1.1.1, v1.2.0, v1.3.0, v1.3.0L2, v1.4.1, v.1.4.1L2
)
and et.gas_used > 10000 -- to ensure the setup call was successful. excludes e.g. setup calls with missing params that fallback
{% if not is_incremental() %}
and et.block_time > TIMESTAMP '2023-09-01' -- for initial query optimisation
jeff-dude marked this conversation as resolved.
Show resolved Hide resolved
{% endif %}
{% if is_incremental() %}
and et.block_time > date_trunc('day', now() - interval '7' day)
jeff-dude marked this conversation as resolved.
Show resolved Hide resolved
{% endif %}
Loading
Loading