-
-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for GCS emulators #177
Add support for GCS emulators #177
Conversation
@@ -18,7 +18,7 @@ gcs_upload( | |||
upload_type = c("simple", "resumable") | |||
) | |||
|
|||
gcs_upload_set_limit(upload_limit = 5000000L) | |||
gcs_upload_set_limit(upload_limit = 5e+06) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find this annoying. Not sure why roxygen2 decided to make this change.
Please do add a note to the NEWS.md and feel free to add as a contributor to DESCRIPTION - thank you! Looks like a useful addition. The rebuild of the documentation is usually done before CRAN launch |
Allow configuring a Google Cloud Storage emulator to be used instead of the real service. This can be achieved by providing a `STORAGE_EMULATOR_HOST` environment variable with the scheme, host and port of the emulator server (e.g.: http://localhost:8080). Besides using a different API endpoint, configuring an emulator also has the effect of lifting the requirement of providing valid authentication credentials. Emulators don't require or even implement authentication. This behavior is similar to the one implemented in official Google Cloud Storage client libraries developed and maintained by Google.
e52a37e
to
68c5019
Compare
Just in case you didn't get a notification from my last change, I have adjusted |
@manuteleco great thank you! I was on holiday |
@manuteleco I am getting some bug reports since this was merged, is it possible this is related? |
Allow configuring a Google Cloud Storage emulator to be used instead of the real service. This can be achieved by providing a
STORAGE_EMULATOR_HOST
environment variable with the scheme, host and port of the emulator server (e.g.: http://localhost:8080).Besides using a different API endpoint, configuring an emulator also has the effect of lifting the requirement of providing valid authentication credentials. Emulators don't require or even implement authentication.
This behavior is similar to the one implemented in official Google Cloud Storage client libraries developed and maintained by Google.
Closes #176
It also has a weak dependency with MarkEdmondson1234/googleAuthR#228. Without that PR in
googleAuthR
, using an emulator will still work ingoogleCloudStorageR
but it will unfortunately require valid credentials for the real GCS.Just like in #175 (comment), I wonder whether I have to build the
pkgdown
documentation here or not. I also wonder what else should I include in this PR (e.g., an entry inNEWS.md
, etc.).Also, be aware that I'm new to R, so feel free to point out any unidiomatic or plain odd code that needs to be fixed.
Finally, for testing I have used https://github.com/fsouza/fake-gcs-server, but it currently requires a few adjustments for it to work with
googleCloudStorageR
: fsouza/fake-gcs-server#1258 and fsouza/fake-gcs-server#1252. With some luck, they will be accepted and merged eventually.I did not test every single GCS operation; it is possible some are still not working with an emulator yet. But I believe this PR already brings value even if it doesn't happen to provide 100% support. Any operation that is still not working could be refined later, once encountered.