Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up ETL directory and Docker image CI #1511

Merged
merged 10 commits into from
Aug 20, 2024

Conversation

johnclary
Copy link
Member

@johnclary johnclary commented Aug 19, 2024

Associated issues

Testing

  1. Before you pull, you probably want to make a copy of your env file at atd-etl/data_model/.env
  2. After you pull, delete the old env filed fro ./atd-etl/cris_import and save your .env file there
  3. Start your local stack
  4. Run the cris import from ./atd-etl/cris_import:
 docker run -it  --rm --env-file .env --network host -v $PWD:/app data_model_cris_import ./cris_import.py --csv  --pdf --s3-download --verbose
  1. Look at the github action changes and see if those look ok 🔍

@johnclary johnclary changed the base branch from master to 18423_remove_views_from_db August 19, 2024 17:04
@johnclary johnclary changed the title Misc loose ends part IV Clean up ETL directory and Docker image CI Aug 19, 2024
Base automatically changed from 18423_remove_views_from_db to data-model-v2 August 19, 2024 17:45
Copy link
Contributor

@mddilley mddilley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran the export locally, and it succeeded. I read through the file updates and deletions. Nothing sticks out to me, and I think the GH action diff looks good too. 🚢 💨

@@ -1 +1,3 @@
env
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll have to remember to delete our existing env files since only .env is ignored at the top level of the project. I just did mine because I'm paranoid that I'll forget and accidentally leak my 1PW token.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah—thank you. i had the luxury of deleting the old directory and renaming the new directory, so didn't encounter this. i just added env to the root .gitignore and i will add note to the test steps to remove that. will also drop a note in slack post-go-live

@johnclary johnclary merged commit 2b73b6f into data-model-v2 Aug 20, 2024
@johnclary johnclary deleted the 18679-misc-loose-ends-iv branch August 21, 2024 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants