Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update how veneur handles aws credentials. #758

Conversation

linuxdynasty
Copy link
Contributor

@linuxdynasty linuxdynasty commented Sep 29, 2019

  • If aws creds are set in config file, try and load those up.
  • If no creds are set, follow the aws sdk golang loading order.

Summary

Updated server.go to use the aws sdk loading order (https://docs.aws.amazon.com/sdk-for-go/api/aws/session/) when the creds are not set in the config.yaml.

The only 2 options that are mandatory are.

  • aws_s3_bucket: string
  • aws_region: string

These parameters are optional.

  • aws_access_key_id string
  • aws_secret_access_key string

The Golang AWS SDK will load up Credentials in the following order. https://docs.aws.amazon.com/sdk-for-go/api/aws/session/

  1. Environment Variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN, AWS_PROFILE, AWS_REGION
  2. Shared Credentials file ~/.aws/credentials
  3. Shared Configuration file (if SharedConfig is enabled) export AWS_SDK_LOAD_CONFIG=1
  4. EC2 Instance Metadata (credentials only).

Motivation

We are deploying Veneur and we do not want to hardcode aws credentials and we would rather use the iam role attached to the instance.


This change is Reviewable

@stripe-ci
Copy link

Gerald Rule: Copy Observability on Veneur, Unilog, Falconer pull requests

cc @stripe/observability
@stripe/observability-stripe

* If aws creds are set in config file, try and load those up.
* If no creds are set, follow the aws sdk golang loading order.
@linuxdynasty
Copy link
Contributor Author

@ChimeraCoder is the plugin system not going to be supported anymore or is this PR incomplete from your opinion and if so, what can I do to make it complete?

@ChimeraCoder
Copy link
Contributor

Really sorry about the delay here. Yes, we do plan to continue supporting this plugin. Thanks for submitting this!

@ChimeraCoder ChimeraCoder merged commit 2220c42 into stripe:master Apr 21, 2020
yanke-stripe pushed a commit that referenced this pull request Jul 3, 2020
* Update how veneur handles aws credentials.

* If aws creds are set in config file, try and load those up.
* If no creds are set, follow the aws sdk golang loading order.

* golang→go

Co-authored-by: Allen Sanabria <allen_sanabria@intuit.com>
Co-authored-by: Aditya Mukerjee <dev@chimeracoder.net>
Co-authored-by: Aditya Mukerjee <ChimeraCoder@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants