Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update status of project wrt Nansen #206

Merged
merged 1 commit into from
Aug 30, 2024
Merged

Update status of project wrt Nansen #206

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,25 @@
Polygon ETL allows you to setup an ETL pipeline in Google Cloud Platform for ingesting Polygon blockchain data
into BigQuery and Pub/Sub. It comes with [CLI tools](/cli) for exporting Polygon data into convenient formats like CSVs and relational databases.


## Status Update (2024-08-29)

### This repo
- Nansen has decided to step back from its role as major contributor to this repo.
- Nansen would like to thank all contributors to this repo, with best wishes for the future.

### Raw data
- Polygon continues to batch load daily to BigQuery dataset `public-data-finance.crypto_polygon`.
- However, streaming to this dataset is currently disabled in Polygon's GCP infra.
- Nansen will no longer be maintaining this dataset; it's up to Polygon what happens next.

### Parsed data
- This repo's [table_definitions folder](./airflow/dags/resources/stages/parse/table_definitions) is now archived. Please do not add table definitions there!
- Nansen continues to batch load daily to BigQuery datasets `blockchain-etl.polygon_*`.
- Nansen will be maintaining this dataset for the foreseeable future.
- If you wish to contribute table definitions, please raise PRs in [nansen-ai/evm-table-definitions](https://github.com/nansen-ai/evm-table-definitions).
Copy link
Member

@medvedev1088 medvedev1088 Aug 31, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link is broken @charlielewisme https://github.com/nansen-ai/evm-table-definitions. Was it supposed to be https://github.com/nansen-ai/evmchain-etl-table-definitions?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. Thanks Evgeny. PR to fix: #208



## Architecture

![polygon_etl_architecture.svg](polygon_etl_architecture.svg)
Expand Down
6 changes: 6 additions & 0 deletions airflow/dags/resources/stages/parse/table_definitions/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
## Status Update (2024-08-29)

- This folder is now archived. Please do not add table definitions here!
- Nansen continues to batch load daily to BigQuery datasets `blockchain-etl.polygon_*`.
- Nansen will be maintaining this dataset for the foreseeable future.
- If you wish to contribute table definitions, please raise PRs in [nansen-ai/evm-table-definitions](https://github.com/nansen-ai/evm-table-definitions).
Loading