Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve ec2_instance_metadata and create ec2_instance_region #349

Merged
merged 4 commits into from
Apr 29, 2021
Merged

Improve ec2_instance_metadata and create ec2_instance_region #349

merged 4 commits into from
Apr 29, 2021

Conversation

omus
Copy link
Member

@omus omus commented Apr 28, 2021 •
edited
Loading

Replaces the _ec2_metadata with the more generalized ec2_instance_metadata. The main change here is using HTTP.jl's connect_timeout which lets this function fail fast if running in and environment where the EC2 metadata is not available.

Additionally I added a ec2_instance_region function which determines the region of an instance when executed on the instance. I originally was planning on using this function as the default for aws_get_region but decided against it. It should still be useful in use cases such as:

global_aws_config(; region=something(AWS.ec2_instance_region(), "us-east-2"))

The one caveat with this change is that it requires a much newer version of HTTP.jl. Note the connect_timeout keyword was available in earlier revisions but had issues. Additionally, the special exception we're using was only available starting in HTTP.jl 0.9.6.

@omus
Copy link
Member Author

omus commented Apr 28, 2021

bors try

bors bot added a commit that referenced this pull request Apr 28, 2021
@bors
Try #349:
413df26
@bors
Copy link
Contributor

bors bot commented Apr 28, 2021

try

Build failed:

@omus
Copy link
Member Author

omus commented Apr 28, 2021

Seems like I missed a patch. I took this opportunity to make the testset nicer too

@omus
Copy link
Member Author

omus commented Apr 28, 2021

bors try

bors bot added a commit that referenced this pull request Apr 28, 2021
@bors
Try #349:
c5d6aac
@bors
Copy link
Contributor

bors bot commented Apr 28, 2021

try

Build failed:

@omus
Copy link
Member Author

omus commented Apr 28, 2021

bors try

bors bot added a commit that referenced this pull request Apr 28, 2021
@bors
Try #349:
ce1371d
@bors
Copy link
Contributor

bors bot commented Apr 28, 2021

try

Build succeeded:

mattBrzezinski
mattBrzezinski reviewed Apr 28, 2021
View reviewed changes
src/AWSCredentials.jl Show resolved Hide resolved
src/AWSCredentials.jl Show resolved Hide resolved
src/AWSCredentials.jl Outdated Show resolved Hide resolved
@ericphanson
Copy link
Member

Does this resolve #287? Or maybe it needs

using this function as the default for aws_get_region

@omus
Copy link
Member Author

omus commented Apr 29, 2021

Does this resolve #287? Or maybe it needs

using this function as the default for aws_get_region

That issue in particular will need support for credential_source = Ec2InstanceMetadata. Ultimately support for that does require access to the instance metadata but there is more work to be done there.

@omus
Copy link
Member Author

omus commented Apr 29, 2021

bors try

bors bot added a commit that referenced this pull request Apr 29, 2021
@bors
Try #349:
53f9ca2
@omus omus mentioned this pull request Apr 29, 2021
Merged
@bors
Copy link
Contributor

bors bot commented Apr 29, 2021

try

Build succeeded:

mattBrzezinski
mattBrzezinski approved these changes Apr 29, 2021
View reviewed changes
Copy link
Member

@mattBrzezinski mattBrzezinski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bors r+

@bors
Copy link
Contributor

bors bot commented Apr 29, 2021

Build succeeded:

@bors bors bot merged commit 446e97d into master Apr 29, 2021
@bors bors bot deleted the cv/ec2-instance-metadata branch April 29, 2021 15:42
bors bot added a commit that referenced this pull request Apr 29, 2021
@bors @omus
Merge #351
86f0424 
351: Use a constant for the default region r=mattBrzezinski a=omus

Mainly just some code refactoring. Could be useful in conjunction with #349 such that you can do:

```julia
global_aws_config(; region=something(AWS.ec2_instance_region(), AWS.DEFAULT_REGION))
```

Co-authored-by: Curtis Vogt <curtis.vogt@gmail.com>
@ericphanson
Copy link
Member

I originally was planning on using this function as the default for aws_get_region but decided against it.

Why is that? I think it would be useful. Right now we set the config in __init__ in some packages to ensure users get a working version, but that can be annoying (maybe they've already setup their config as they want it and don't want loading a package to mess with it). But I think if getting the region from the ec2 region was automatic then that would make it less necessary (and ensure that missing an env variable with the region in say a Dockerfile doesn't mean you get runtime errors from your job).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mattBrzezinski mattBrzezinski mattBrzezinski approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@omus @ericphanson @mattBrzezinski