New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automate GBIF filtering, triggered by new GBIF data, including name generation #77

Open

zzeppozz opened this issue Mar 8, 2024 · 0 comments

Assignees

Labels

Analyst enhancement

Contributor

zzeppozz commented Mar 8, 2024 •

edited

Loading

New GBIF data in AWS should trigger events in Redshift:

generate dataset date, used in GBIF data path and SpNetwork name
mount full GBIF dataset
filter to appropriate dataset for Sp Network
dismount full GBIF dataset
summarize data contents by dataset to tables
write to S3
delete previous month's SpNetwork dataset and summary data tables

zzeppozz self-assigned this

zzeppozz added Analyst enhancement labels

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment