Uploading a file with storage and a different key is ... confusing?

[jgeewax]

I think the following code works fine on GCE:

from gcloud import storage
bucket = storage.get_bucket('my-bucket')
bucket.upload_file('local.png', 'remote.png')

Now I want to change the credentials, since I'm not on GCE/GAE. How do I do that? Could it be...

from gcloud import storage
storage.set_credentials_json('path/to/creds.json')
bucket = storage.get_bucket('my-bucket')
bucket.upload_file('local.png', 'remote.png')

I couldn't find this in the docs (though I'll admit I didn't scour it to death)... so feel free to close and link if you know where this lives...

[dhermes]

@jgeewax #591 and #633 are still open, but we wanted to get storage "stable" before doing them. Auth / defaults in storage are still not stable due to #728.

I'd like to close this and just focus on #633.

---

To answer your question, the recommended way to get auth is:

• GCE or GAE, do nothing (library picks it up)
• Elsewhere, set GOOGLE_APPLICATION_CREDENTIALS env. var. to path of JSON key
• Just playing on local machine, run gcloud auth login and library picks it up

[jgeewax]

#633 is important -- totally agree, but I still want a way to specify a key path in code. Without this method, we're effectively saying that you should import sys.environ and update a special item on that dictionary rather than providing a method that let's you specify a key.

And what if we want to use multiple keys (say I have two accounts and want to upload two files to two different projects) ?

[dhermes]

• People don't set environment variables in code, they do it in the environment (so we're not asking people to touch sys.environ)
• Using multiple credentials is possible in storage just as we've discussed it's possible in datastore. There is a library-wide concept of Connection (implemented in each sub-package with API specific features) and both explicit and implicit connections can be used whenever an HTTP request is needed. To explicitly create a Connection, a set of credentials from oauth2client can be used. We provide methods in gcloud.credentials to make this easy. These are already "documented" though below the fold. Hence the remaining piece of the puzzle is adding this in an auth document (Add auth doc; Document 3LO via "gcloud login" #633).

[jgeewax]

Got it, looks like the code I'd have to write is....

from gcloud import credentials
from gcloud import storage

creds = credentials.get_for_service_account('/path/to/key.json', scope=???)
conn = storage.Connection(credentials=creds)
storage.set_default_connection(conn)
bucket = storage.get_bucket('my-bucket')
bucket.upload_file('local.png', 'remote.png')

Is that right?

If so, it seems a bit verbose (and I don't know exactly what to put down as the scope). Probably worth shortcutting that to a single line, no?

from gcloud import storage

storage.set_credentials_json('path/to/creds.json')
bucket = storage.get_bucket('my-bucket')
bucket.upload_file('local.png', 'remote.png')

[dhermes]

Yes, except for

• get_for_service_account -> get_for_service_account_json
• You don't need the scope when creating the credentials. We use connection.get_scoped_connection to add credentials within each subpackage. (Though I now realize we could check this in Connection constructors)
• It's not necessary to construct a connection and then set it as the default. Though in the "two at once" scenario, you would want them explicitly. Instead of setting the default you'd pass it in

storage.get_bucket('my-bucket', connection=conn)

---

As for enabling storage.set_credentials_json, it's already possible via setting the environment variable. This is just a more declarative / explicit synonym.

We may consider adding optional keywords to get_connection(), but we tried it before and were stymied by the number of possible accounts that could be used.

[dhermes]

No longer relevant in Client context