Uploading compressed tarball file to S3 #250
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
StuAA78
requested changes
Jun 1, 2023
There was a problem hiding this comment.
This is great! I noticed we're using readFileSync which we'd be better off avoiding if possible. The "define filename as a const" comment is just a small thing we could maybe refactor. The comment about _uploadToBucket() is more of a nice to have and it's a little dependent on what was decided on about error handling so I'm not going to insist we make changes to it at the moment if it needs any more than a few mins of work.
Cruikshanks
requested changes
Jun 1, 2023
Cruikshanks
requested changes
Jun 1, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://eaflood.atlassian.net/browse/WATER-4019
This pull request is just a small part of a larger project that involves exporting all our database schemas, converting them into CSV files, and uploading them to our Amazon S3 bucket.
This PR's primary focus is to upload a single compressed tarball file to our S3 bucket rather then individual table files.
Additionally, it aims to improve error handling throughout the service, ensuring that errors are only handled at the top-level service and appropriately addressed without interrupting the subsequent schema export process.