Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle S3 credential expiration more gracefully #354

Merged
merged 6 commits into from
Nov 30, 2023

Conversation

jrbourbeau
Copy link
Collaborator

Closes #353

Copy link

github-actions bot commented Nov 14, 2023

Binder 👈 Launch a binder notebook on this branch for commit 64c0976

I will automatically update this comment whenever this PR is modified

Binder 👈 Launch a binder notebook on this branch for commit a33bf3a

Binder 👈 Launch a binder notebook on this branch for commit ab70e65

Binder 👈 Launch a binder notebook on this branch for commit ce741a8

Binder 👈 Launch a binder notebook on this branch for commit ad85cd4

Binder 👈 Launch a binder notebook on this branch for commit 6c342a2

earthaccess/store.py Outdated Show resolved Hide resolved
tests/unit/test_store.py Show resolved Hide resolved
@jrbourbeau
Copy link
Collaborator Author

@MattF-NSIDC @betolink I confirmed the changes here work as expected in practice (i.e. no longer getting the permissions errors reported in the #353)

@MattF-NSIDC
Copy link

It may realistically be a while before I can review this! 😢

mfisher87
mfisher87 previously approved these changes Nov 22, 2023
Copy link
Collaborator

@mfisher87 mfisher87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! I have a couple thoughts about readability, but no concerns about the behavior at the moment.

I don't think the suggestions I made are significant enough to warrant a second review, so if you're ready to merge after going through them: 🚀

earthaccess/store.py Show resolved Hide resolved
earthaccess/store.py Show resolved Hide resolved
earthaccess/store.py Show resolved Hide resolved
earthaccess/store.py Show resolved Hide resolved
earthaccess/store.py Outdated Show resolved Hide resolved
earthaccess/store.py Show resolved Hide resolved
earthaccess/store.py Outdated Show resolved Hide resolved
tests/unit/test_store.py Show resolved Hide resolved
@betolink
Copy link
Member

related to this, just for reference, Alex Mandel from DevSeed mentioned this piece of code that handles more or less the same for the MAAP project https://github.com/MAAP-Project/gedi-subsetter/blob/58c87ab907bc3b121510eeabdeb2b8e63c7df999/src/gedi_subset/maapx.py#L55-L61

@jrbourbeau jrbourbeau mentioned this pull request Nov 29, 2023
Co-authored-by: Matt Fisher <mfisher87@gmail.com>
@jrbourbeau
Copy link
Collaborator Author

mentioned this piece of code that handles more or less the same for the MAAP project

Nice, thanks for pointing us to that @betolink. Good to know for future reference.

@jrbourbeau
Copy link
Collaborator Author

@mfisher87 I've addressed or responded to all your review comments (thank you for taking the time to review this PR). There are a few minor, subjective points where I think we have differing opinions / styles -- let me know if any of those are blocking comments.

@MattF-NSIDC
Copy link

@jrbourbeau thanks! I don't feel any of those differing opinions / styles should be blockers, marked all as resolved! I think this is ready to go 💯

@@ -209,8 +209,15 @@ def get_s3fs_session(
"parameters must be specified. "
)

if concept_id is not None:
provider = self._derive_concept_provider(concept_id)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

@jrbourbeau jrbourbeau merged commit cbc3bc4 into nsidc:main Nov 30, 2023
7 checks passed
@jrbourbeau jrbourbeau deleted the cache-s3-creds branch November 30, 2023 22:03
@jrbourbeau jrbourbeau mentioned this pull request Dec 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Not sensitive to temporary S3 credentials expiring
4 participants