NoCredentialProviders: no valid providers in chain #962

Smana · 2016-11-21T10:06:30Z

Hello guys,

I'm currently trying kops and i'm not able to use a profile listed in ~/.aws directory.
I already had a look to a similar past issue #855.
But my credentials file is not a symlink.

running the aws cli works fine

aws s3 ls s3://kops-${mydomain} && echo $?
0

But not with kops

kops create cluster --cloud=aws --zones=eu-west1-a cluster0.${mydomain}
error reading cluster configuration "cluster0.${mydomain}": error reading configuration file s3://kops-${mydomain}/cluster0.${mydomain}/config: error getting location for S3 bucket "kops-${mydomain}": NoCredentialProviders: no valid providers in chain. Deprecated.

env | egrep '(AWS_PROFILE|KOPS)'
AWS_PROFILE=${myprofile}
KOPS_STATE_STORE=s3://kops-${domain}

How can i solve that please ?

The text was updated successfully, but these errors were encountered:

krisnova · 2016-11-21T14:53:02Z

What does your aws config look like?

I would be willing wager that your aws config is default to some region, while kops defaults to us-east-1 #L41

More information on your aws config can be found here

There is a PR out to override this behavior #961 with an env var, although I think we need to give the util/pkg/vfs/s3context.go a once over to correct this and #961

I think a more robust patch is due here.. I will see about adding it...

Detection steps

Can you please add a bucket to us-east-1 called kops-${domain} and try again? Just wondering if my hunch here is correct.. If so the patch will be pretty straightforward

Smana · 2016-11-21T16:19:59Z

Hi @kris-nova,

Indeed my default region is eu-west-1.
I tried to create a bucket in the US but i still get the same error.

Then i changed my default region to us-east-1 but it didn't help

aws configure get region
us-east-1

kops create cluster --cloud=aws --zones=eu-west1-a cluster0.${mydomain} -v=3
I1121 17:13:29.030311    4849 s3context.go:61] Querying S3 for bucket location for "kops-k8s-us-east-1"

error reading cluster configuration "cluster0.${mydomain}": error reading configuration file s3://kops-k8s-us-east-1/cluster0.${mydomain}/config: error getting location for S3 bucket "kops-k8s-us-east-1": NoCredentialProviders: no valid providers in chain. Deprecated. 
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

ghost · 2016-12-28T06:56:48Z

I am having the same issue,

ubuntu@ip-private-ip:~$ kops delete cluster kubernetes1.in
error reading cluster configuration "kubernetes1.in": error reading configuration file s3://examplekubernetes.in/kubernetes1.in/config: error getting location for S3 bucket "examplekubernetes.in": NoCredentialProviders: no valid providers in chain. Deprecated. 
    For verbose messaging see aws.Config.CredentialsChainVerboseErrors

krisnova · 2016-12-28T13:14:51Z

Sorry this issue must have gotten lost in my inbox @Smana

Could you and @voyalab mention what versions of kops you are running? Also were either of you able to get this working on your own?

justinsb · 2016-12-28T17:55:47Z

The default region shouldn't matter - the only time it's a problem is if you don't have access to us-east-1 (i.e. if you're running with a china/govcloud account)

Can you (both) confirm that you're setting AWS_PROFILE, but not AWS_DEFAULT_PROFILE, AWS_ACCESS_KEY_ID or AWS_SECRET_KEY etc. In short, AWS_PROFILE should be the only AWS env var set.

@voyalab can you confirm that aws ec2 --region us-east-1 describe-regions works there.

bacongobbler · 2017-01-05T19:20:14Z

Hey, just tacking on here that I am seeing this as well. Just migrating from kube-up (finally) and I'm seeing this as well.

><> kops create cluster --zones us-west-1a bacongobbler

error reading cluster configuration "bacongobbler": error reading configuration file s3://kops-bacongobbler/bacongobbler/config: error getting location for S3 bucket "kops-bacongobbler": NoCredentialProviders: no valid providers in chain. Deprecated. 
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors

I can confirm that aws ec2 --region us-east-1 describe-regions works for me and I can see both us-east-1 and us-west-1. Dropping into aws ec2 describe-availability-zones --region us-west-1 I can see us-west-1a as available. The kops-bacongobbler bucket exists as well.

AWS_PROFILE is not set on my computer, only AWS_S3_BUCKET which I was using previously with kube-up:

><> env | grep AWS
AWS_S3_BUCKET=bacongobbler-k8s

Running with v1.4.4

EDIT: also, aws config:

[default]
region = us-west-1
aws_access_key_id = REDACTED
aws_secret_access_key = REDACTED
s3 =
    signature_version = s3v4

bacongobbler · 2017-01-05T19:30:19Z

Found out that it was somewhat of a dupe of #855 (comment). Permissions on ~/.aws/config was 600 but was not a symlink. Everything works now after setting AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY!

chrislovecnm · 2017-01-07T02:49:43Z

Can we close?

bacongobbler · 2017-01-07T02:59:04Z

Not sure if this resolves the problem for anyone else but sure I guess

chrislovecnm · 2017-01-08T12:11:09Z

I am getting an error where kops will work with the env variables and not the config file. I am building off of master, and the account is running TFA.

Smana · 2017-01-13T08:18:17Z

I'll test it again and i'll keep you updated.

Smana · 2017-01-13T08:54:56Z

I still get the same error. Should i open a new issue ?

kops version
Version 1.4.4

kops create cluster --node-count 4 --zones eu-west-1a,eu-west-1b --dns-zone ${MYZONE} --node-size t2.medium --name ${CLUSTER_NAME}

error reading cluster configuration "${CLUSTER_NAME}": error reading configuration file s3://${S3_BUCKET}/${CLUSTER_NAME}/config: error getting location for S3 bucket "${S3_BUCKET}": NoCredentialProviders: no valid providers in chain. Deprecated. 
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors

Note: i'm using a MFA authentication

Smana · 2017-01-13T08:56:11Z

#226

ghost · 2017-01-14T16:30:27Z

I was using kops 1.4.1 previously. I solved that problem by providing necessary IAM roles. Also I upgraded version of KOPS 1.4.4 . @justinsb @kris-nova Unfortunately , There is no documentation for IAM roles.

mayurpatil11 · 2018-08-25T15:44:41Z

Can anybody help me withbelow error message: Since I am new to K8s deployment I might be doing silly mistake please bare me with it.
(I am trying to install K8s cluster on AWS ubuntu server using Kops commands.

error reading cluster configuration "dev.k8scluster.com": error reading s3://storage.dev.k8ss.com/dev.k8scluster.com/config: Unable to list AWS regions: NoCredentialProviders: no valid providers in chain
caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
SharedCredsLoad: failed to load profile, .
EC2RoleRequestError: no EC2 instance role found
caused by: EC2MetadataError: failed to make EC2Metadata request
caused by:

<title>404 - Not Found</title>

debu99 · 2018-12-01T02:58:59Z

you must export the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, and it is not helpful even you already set it on ~/.aws/config

fernandrone · 2019-03-12T19:45:32Z

I'm having the same issue. Essentially, when using AWS_PROFILE I'm forced to set both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (as @debu99 said); even though my credentials are configured on ~/.aws/credentials, kops will fail.

Example, here's my config:

$ cat ~/.aws/config
[default]
output = json
region = us-east-1

[profile admin]
role_arn = <redacted>/Admin
source_profile = default
region = us-east-1

$ cat ~/.aws/credentials
[default]
aws_access_key_id=<redacted>
aws_secret_access_key=<redacted>

With this configuration and just AWS_PROFILE=admin set, aws ec2 --region us-east-1 describe-regions works fine. So the profile configuration is ok.

But kops fails:

$ AWS_PROFILE=admin kops get ig
error fetching cluster "<redacted>": error reading cluster configuration "<redacted>": error reading s3://<redacted>/config: Unable to list AWS regions: NoCredentialProviders: no valid providers in chain
caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
SharedCredsLoad: failed to load profile, admin.
EC2RoleRequestError: no EC2 instance role found
caused by: RequestError: send request failed
caused by: Get http://169.254.169.254/latest/meta-data/iam/security-credentials/: dial tcp 169.254.169.254:80: connect: no route to host

However, if I set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, it works:

$ AWS_PROFILE=admin AWS_ACCESS_KEY_ID=<redacted> AWS_SECRET_ACCESS_KEY=<redacted> kops get ig

In case it's relevant:

$ kops version
Version 1.11.0 (git-2c2042465)

superma58 · 2021-12-17T14:42:35Z

I think the SDK doesn't load all AWS shared config(~/.aws/config) as default, which causes error 'EnvAccessKeyNotFound: failed to find credentials in the environment'.
So I fixed this issue by passing environment AWS_SDK_LOAD_CONFIG=true. This environment make SDK to load the all AWS shared config. Don't need to set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

borelsaffo · 2023-01-03T20:02:50Z

root@ip-172-31-85-125:/home/ubuntu# kops create cluster --name EKS3 --state=s3://borel1 --zones us-east-1

error reading cluster configuration "EKS3": error reading s3://borel1/EKS3/config: Unable to list AWS regions: NoCredentialProviders: no valid providers in chain
caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
SharedCredsLoad: failed to load profile, .
EC2RoleRequestError: no EC2 instance role found
caused by: EC2MetadataError: failed to make EC2Metadata request
status code: 404, request id:
caused by:

<title>404 - Not Found</title>

404 - Not Found

root@ip-172-31-85-125:/home/ubuntu# env | egrep '(AWS_PROFILE|KOPS)'
KOPS_STATE_STORE=s3://borel1
root@ip-172-31-85-125:/home/ubuntu#

olemarkus · 2023-01-10T09:02:27Z

@borelsaffo please file a new bug following the template.

krisnova added the Help Wanted! label Nov 28, 2016

justinsb added this to the backlog milestone Dec 28, 2016

chrislovecnm closed this as completed Jan 7, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NoCredentialProviders: no valid providers in chain #962

NoCredentialProviders: no valid providers in chain #962

Smana commented Nov 21, 2016

krisnova commented Nov 21, 2016

Smana commented Nov 21, 2016

ghost commented Dec 28, 2016 •

edited by voyalab-temp

Loading

krisnova commented Dec 28, 2016

justinsb commented Dec 28, 2016

bacongobbler commented Jan 5, 2017 •

edited

Loading

bacongobbler commented Jan 5, 2017 •

edited

Loading

chrislovecnm commented Jan 7, 2017

bacongobbler commented Jan 7, 2017

chrislovecnm commented Jan 8, 2017

Smana commented Jan 13, 2017

Smana commented Jan 13, 2017

Smana commented Jan 13, 2017

ghost commented Jan 14, 2017

mayurpatil11 commented Aug 25, 2018

debu99 commented Dec 1, 2018

fernandrone commented Mar 12, 2019 •

edited

Loading

superma58 commented Dec 17, 2021 •

edited

Loading

borelsaffo commented Jan 3, 2023

olemarkus commented Jan 10, 2023

NoCredentialProviders: no valid providers in chain #962

NoCredentialProviders: no valid providers in chain #962

Comments

Smana commented Nov 21, 2016

krisnova commented Nov 21, 2016

Detection steps

Smana commented Nov 21, 2016

ghost commented Dec 28, 2016 • edited by voyalab-temp Loading

krisnova commented Dec 28, 2016

justinsb commented Dec 28, 2016

bacongobbler commented Jan 5, 2017 • edited Loading

bacongobbler commented Jan 5, 2017 • edited Loading

chrislovecnm commented Jan 7, 2017

bacongobbler commented Jan 7, 2017

chrislovecnm commented Jan 8, 2017

Smana commented Jan 13, 2017

Smana commented Jan 13, 2017

Smana commented Jan 13, 2017

ghost commented Jan 14, 2017

mayurpatil11 commented Aug 25, 2018

debu99 commented Dec 1, 2018

fernandrone commented Mar 12, 2019 • edited Loading

superma58 commented Dec 17, 2021 • edited Loading

borelsaffo commented Jan 3, 2023

404 - Not Found

olemarkus commented Jan 10, 2023

ghost commented Dec 28, 2016 •

edited by voyalab-temp

Loading

bacongobbler commented Jan 5, 2017 •

edited

Loading

bacongobbler commented Jan 5, 2017 •

edited

Loading

fernandrone commented Mar 12, 2019 •

edited

Loading

superma58 commented Dec 17, 2021 •

edited

Loading