Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core library does not include init in root path #4247

Closed
Downchuck opened this issue Oct 24, 2017 · 16 comments
Closed

Core library does not include init in root path #4247

Downchuck opened this issue Oct 24, 2017 · 16 comments
Assignees

Comments

@Downchuck
Copy link

Downchuck commented Oct 24, 2017

There is clearly a hierarchy in this package, for example "google.cloud.storage" is a particular package that includes the google.cloud core library. But there is no upstream "google" library, and so there is no associated google/__init__.py file created, which can lead to a painful debugging situation where simple code like "import google.cloud.storage" fails.

In depth:

This is a corner case -- but allowing for that: the wheel packages for Google Cloud Python are setup to use subfolders, which is relatively unique in Python packaging. So these packages are all underneath the "google/" folder. Unfortunately, there is no root package -- because the Google Cloud Core is not actually a root package, it too is a subfolder.

The consequence is that there is no google/__init__.py and there is no package that would own that file as a distributable wheel. This has some unfortunate side effects when attempting to do an import google.cloud.storage or similar when using Spark.

The workaround is simply to touch that file when packaging dependencies.

@dhermes
Copy link
Contributor

dhermes commented Oct 24, 2017

@Downchuck It seems you are having an issue with namespace packages in your install.

The real problem is that google is a namespace that many other packages outside of our purview own. And if each of those packages tried to over-write google/__init__.py, bad things would happen.

There should not be a google/__init__.py file, but there should be .pth files, e.g.

$ ls -1 site-packages/*pth
site-packages/googleapis_common_protos-1.5.3-py3.6-nspkg.pth
site-packages/google_auth-1.1.1-py2.7-nspkg.pth
site-packages/google_cloud_core-0.27.1-py3.6-nspkg.pth
site-packages/google_cloud_storage-1.4.0-py3.6-nspkg.pth
site-packages/google_resumable_media-0.3.0-py3.6-nspkg.pth
site-packages/protobuf-3.4.0-py3.6-nspkg.pth
site-packages/pytest-cov.pth

@dhermes
Copy link
Contributor

dhermes commented Oct 24, 2017

@Downchuck I am going to pre-emptively close since the namespace packages are:

  • Our only choice
  • An officially documented (and PEP-accepted) part of Python packaging
  • Working "as intended"

If you'd like to share some stacktraces, we are happy to help troubleshoot within your environment. (I'm not familiar with what sort of Python interpreters are used for large Spark jobs.)

@dhermes dhermes closed this as completed Oct 24, 2017
@Downchuck
Copy link
Author

Has PEP closed out the concept of a root package?

@tseaver
Copy link
Contributor

tseaver commented Oct 24, 2017

See also: pypa/setuptools#912

@dhermes
Copy link
Contributor

dhermes commented Oct 24, 2017

Has PEP closed out the concept of a root package?

I'm sorry I don't understand what you mean.

@Downchuck
Copy link
Author

Downchuck commented Oct 24, 2017

I get that it'd need a root package if a root __init__.py file is going to be created. That's why I'm saying, there ought to be a root package in the dependency chain, even if it's just a single file. This is more an issue of "pain" than it is of anything else. Going with the idea that "With google-cloud-python we try to make authentication as painless as possible" is a worthwhile goal.

You can see in that thread you posted the pits of suffering us poor developers fall into, when backed into the corner of otherwise elegant solutions.

Here's what I'm doing, so I can use Spark's addPyFile

python -m pip wheel --wheel-dir ./google-cloud-storage google-cloud-storage
cd google-cloud-storage
for x in *.whl; do unzip -d all -o -u $x; done;
cd all
touch google/__init__.py
touch google/cloud/__init__.py
zip -r ../google-cloud-storage-py27-none-any.all.zip .

That's not a big deal, but it meant a lot of pain, to figure out that I needed to touch a couple of files.
This is in contrast with every other package I've worked with in this system, where that step wasn't needed.

@dhermes
Copy link
Contributor

dhermes commented Oct 24, 2017

There is no need for a "root package". (I'm not sure what you mean when you say "root package".)


that I needed to touch a couple of files

But you don't need to touch any files, you need an interpreter that supports namespace packages (i.e. can parse .pth files). This shouldn't be an issue, as namespace packages have been around since 2012.

@dhermes
Copy link
Contributor

dhermes commented Oct 24, 2017

It's likely that Spark's addPyFile is only intended for simple modules / self-contained .zip files. Passing in a wheel seems to subvert this usage (i.e. a wheel file assumes it will be installed by pip, which is what will add the .pth files).

Can you point to some documentation for addPyFile?

@duggelz
Copy link

duggelz commented Oct 24, 2017

addPyFile implementation is here, which appears to be the best documentation as well:

https://github.com/apache/spark/blob/7c7266208a3be984ac1ce53747dc0c3640f4ecac/python/pyspark/context.py#L846

It was written some time ago, before namespace packages and wheels existed. It could be changed to do so, which would be a feature request for the Spark project. Otherwise, we'd have to package the entire transitive closure of google-cloud-python as an .egg or .pyz (or .par).

@tseaver
Copy link
Contributor

tseaver commented Oct 25, 2017

[addPyFile] was written some time ago, before namespace packages and wheels existed.

Namespace packages have existed much longer than that: they have been present since setuptools 0.5a9, released in June 2005.

@Downchuck
Copy link
Author

Some other related docs, such as zipimport; potentially addressed in a later PEP in Python 3, like importlib:
https://docs.python.org/2/library/zipimport.html
https://docs.python.org/3/library/importlib.html

When I say root package, I mean a python package that represents the root namespace "google" so that only one package is writing to that init file. It seems that the cloud core would write an init to "google/cloud/" for example. So google core would write to "google/".

Regardless of outcome, I hope this conversation helps if someone else hits the same issue I did. It took some time to figure out.

@dhermes
Copy link
Contributor

dhermes commented Oct 26, 2017

When I say root package, I mean a python package that represents the root namespace "google" so that only one package is writing to that init file. It seems that the cloud core would write an init to "google/cloud/" for example. So google core would write to "google/".

That's not quite how namespace packages work. google-cloud-core puts code in google.cloud, so there is nothing to place in the google namespace package (other than the google.cloud subpackage).

As @duggelz notes, your best bet is to probably package up all of the files in a self-contained zip. You'll be better off doing this by installing into a virtualenv and then zipping what's installed. (Due to the lack of an obvious way to get Spark to support namespace packages, you will still need to touch the __init__.py files as a workaround).

$ virtualenv venv
$ venv/bin/pip install google-cloud-storage
$ touch venv/lib/python3.6/site-packages/google/__init__.py
$ touch venv/lib/python3.6/site-packages/google/cloud/__init__.py
$ zip -r all_of_it.zip venv/lib/python3.6/site-packages/

(i.e. I'm suggesting you use something other than the wheel for your zip).

Sorry for the pain this has caused you.

@zafields
Copy link

zafields commented Nov 7, 2017

@Downchuck I am having a similar experience as you (linked above), and it turns out the root cause is pip and how it is installed. Debian uses dist-packages instead of site-packages as its package folder. For some reason, the Google packages don't seem to recognize this directory without having the __init__.py file.

Are you using Debian? You can see whether you are using dist-packages or site-packages by calling

pip --version

and checking which of the two is reported by the URI in the response.

I'm still trying to figure it out, and I would love to see the details from your pip install/setup.

@dhermes
Copy link
Contributor

dhermes commented Nov 7, 2017

@zafields This is not really an accurate description of the issue. The problem is mixing a system installed pip with a non-system installed Python. I recommend you use get-pip with a built Python or you just use the system installed python (which is also the 4 years old 2.7.6 on the 3 years old OS Ubuntu 14.04).

@zafields
Copy link

zafields commented Nov 7, 2017

@dhermes That sounds like a really interesting option. Can you demonstrate how to download/use get-pip.py script securely, as opposed to running a script from the internet? Does it have a hash? I didn't see one listed at https://packaging.python.org/tutorials/installing-packages/

@dhermes
Copy link
Contributor

dhermes commented Nov 7, 2017

You can just download it locally (e.g. from https://github.com/pypa/get-pip or from the URL) and then manually include that file (rather than downloading) when building your Docker image.

parthea pushed a commit that referenced this issue Oct 21, 2023
* Add new "quickstart" samples [(#547)](GoogleCloudPlatform/python-docs-samples#547)

* Quickstart tests [(#569)](GoogleCloudPlatform/python-docs-samples#569)

* Add tests for quickstarts
* Update secrets

* Add translate samples [(#574)](GoogleCloudPlatform/python-docs-samples#574)

* Generate most non-appengine readmes

Change-Id: I3779282126cdd05b047194d356932b9995484115

* Update requirements.txt

* added model in the sample [(#650)](GoogleCloudPlatform/python-docs-samples#650)

* added with and without model separately [(#657)](GoogleCloudPlatform/python-docs-samples#657)

* Translate API no longer requires an API key. [(#659)](GoogleCloudPlatform/python-docs-samples#659)

* Auto-update dependencies. [(#715)](GoogleCloudPlatform/python-docs-samples#715)

* Changes default model to premium [(#749)](GoogleCloudPlatform/python-docs-samples#749)

* Updates readme template to show credential auth instead of api key. [(#802)](GoogleCloudPlatform/python-docs-samples#802)

* Updates translate version to match speech and language. [(#843)](GoogleCloudPlatform/python-docs-samples#843)

* Translate update [(#886)](GoogleCloudPlatform/python-docs-samples#886)

* Remove cloud config fixture [(#887)](GoogleCloudPlatform/python-docs-samples#887)

* Remove cloud config fixture

* Fix client secrets

* Fix bigtable instance

* Unicode for translation

* Adds check for translated text

* Unicode for translation

* Unicode for translation

* Unicode for translation

* missing newline

* Adds six for 2/3 compatibility

* Changes encoding style.

* Fix unicode

* renaming of the product to Google Translation API [(#910)](GoogleCloudPlatform/python-docs-samples#910)

* Update translate readme

* Re-generate all readmes

* Fix README rst links [(#962)](GoogleCloudPlatform/python-docs-samples#962)

* Fix README rst links

* Update all READMEs

* Auto-update dependencies. [(#1004)](GoogleCloudPlatform/python-docs-samples#1004)

* Auto-update dependencies.

* Fix natural language samples

* Fix pubsub iam samples

* Fix language samples

* Fix bigquery samples

* Auto-update dependencies. [(#1055)](GoogleCloudPlatform/python-docs-samples#1055)

* Auto-update dependencies.

* Explicitly use latest bigtable client

Change-Id: Id71e9e768f020730e4ca9514a0d7ebaa794e7d9e

* Revert language update for now

Change-Id: I8867f154e9a5aae00d0047c9caf880e5e8f50c53

* Remove pdb. smh

Change-Id: I5ff905fadc026eebbcd45512d4e76e003e3b2b43

* Auto-update dependencies. [(#1093)](GoogleCloudPlatform/python-docs-samples#1093)

* Auto-update dependencies.

* Fix storage notification poll sample

Change-Id: I6afbc79d15e050531555e4c8e51066996717a0f3

* Fix spanner samples

Change-Id: I40069222c60d57e8f3d3878167591af9130895cb

* Drop coverage because it's not useful

Change-Id: Iae399a7083d7866c3c7b9162d0de244fbff8b522

* Try again to fix flaky logging test

Change-Id: I6225c074701970c17c426677ef1935bb6d7e36b4

* Update all generated readme auth instructions [(#1121)](GoogleCloudPlatform/python-docs-samples#1121)

Change-Id: I03b5eaef8b17ac3dc3c0339fd2c7447bd3e11bd2

* Added Link to Python Setup Guide [(#1158)](GoogleCloudPlatform/python-docs-samples#1158)

* Update Readme.rst to add Python setup guide

As requested in b/64770713.

This sample is linked in documentation https://cloud.google.com/bigtable/docs/scaling, and it would make more sense to update the guide here than in the documentation.

* Update README.rst

* Update README.rst

* Update README.rst

* Update README.rst

* Update README.rst

* Update install_deps.tmpl.rst

* Updated readmegen scripts and re-generated related README files

* Fixed the lint error

* Auto-update dependencies. [(#1186)](GoogleCloudPlatform/python-docs-samples#1186)

* Fixed failed tests on Kokoro (Spanner + Translate) [(#1192)](GoogleCloudPlatform/python-docs-samples#1192)

* Fixed failed tests on Kokoro (Spanner + Translate)

* Update quickstart_test.py

* Added "Open in Cloud Shell" buttons to README files [(#1254)](GoogleCloudPlatform/python-docs-samples#1254)

* Auto-update dependencies. [(#1377)](GoogleCloudPlatform/python-docs-samples#1377)

* Auto-update dependencies.

* Update requirements.txt

* Regenerate the README files and fix the Open in Cloud Shell link for some samples [(#1441)](GoogleCloudPlatform/python-docs-samples#1441)

* Update READMEs to fix numbering and add git clone [(#1464)](GoogleCloudPlatform/python-docs-samples#1464)

* Add translate region tags [(#1488)](GoogleCloudPlatform/python-docs-samples#1488)

* Add region tags

* Added end region tags

* Linting errors fixed

* Include the comma in the translation [(#1787)](GoogleCloudPlatform/python-docs-samples#1787)

* Auto-update dependencies. [(#1980)](GoogleCloudPlatform/python-docs-samples#1980)

* Auto-update dependencies.

* Update requirements.txt

* Update requirements.txt

* Translation v3beta1 samples [(#2084)](GoogleCloudPlatform/python-docs-samples#2084)

* Add in progress beta snippets

Change-Id: I2cd8ddc2307a8e40d56ce7e493749dc05c34d164

* Add google-cloud-storage dependency

Change-Id: Iff7bc9b2c82b1e829580a3d4ad628087dbeee8be

* Non-'global' location required for BatchTranslateText

Change-Id: I5198aa6368a088e8f5ee295dc55a5e9e4ca8f494

* Upgrade google-cloud-translate to 1.4.0

1.4.0 includes the new v3beta1 alongside V2

Change-Id: I5adfe78ea7e78d84678db343cd84516e3d05491f

* Update Translate samples

You can now provide your own glossary ID

The tests now run within a randomly created bucket (deleted after each
test)

Change-Id: I5cb2680cd0e9e43c85932a6a0dc19e6fab5008c5

* pytest.fixture for random test bucket

Change-Id: I8e816ed4c95a6235347a29849044b4cab02d40a8

* flake8 spec fixes

Change-Id: I4932bcf856a9498b01d9661c90c6b45ee2958ee1

* Added pytest fixture for creating glossary (WIP)

Change-Id: Iddb5ecbf0eefb9efd2243dc4bc56b585102e9351

* Add assertions, remove placeholder TODOs

Change-Id: If1eb20bca5bfcc87dd0652d5488b2188afa626af

* fixing translate-with-glossary bug [(#2323)](GoogleCloudPlatform/python-docs-samples#2323)

* Translate beta samples fix [(#2327)](GoogleCloudPlatform/python-docs-samples#2327)

* fixing translate-with-glossary bug

* tests passing

* reverting to python3 compatibility

* snippets test fix

* Using glossaries with tts and vision tutorial sample code [(#2325)](GoogleCloudPlatform/python-docs-samples#2325)

* fixing translate-with-glossary bug

* initial commit

* adding resources

* adding more resources

* glossary accomodates upper case words

* finished hybrid glossaries tutorial sample code

* Revert "fixing translate-with-glossary bug"

This reverts commit 6a9f7ca3f68239a862106fcbcd9c73649ce36c77.

* lint fix for tests. TODO src lint fix

* lint

* it's the final lint-down

* adding README

* implementing @nnegrey's feedback

* lint

* lint

* extracting files from cloud-client

* lint comment test

* fixing comments per @beccasaurus

* removing redundant directory

* implementing @nnegrey's feedback

* lint

* lint

* handling glossary-already-exists exception

* lint

* adding ssml functionality

* fixing imports per @nnegrey

* fixed import comment

* more specific exceptions import

* removing period from copyright

* fix: refactored MP3 file creation test for Hybrid glossaries samples [(#2379)](GoogleCloudPlatform/python-docs-samples#2379)

* fix: refactored MP3 file creation test

* fix: lint

* Fix variable names in comments [(#2400)](GoogleCloudPlatform/python-docs-samples#2400)

* Adds updates for samples profiler ... vision [(#2439)](GoogleCloudPlatform/python-docs-samples#2439)

* Update v2 samples to explicitly use v2 library [(#2498)](GoogleCloudPlatform/python-docs-samples#2498)

* fix: translate test [(#2671)](GoogleCloudPlatform/python-docs-samples#2671)

* fix: translate test

* Add unicode formatting

* automl: add natural language sentiment analysis ga samples [(#2677)](GoogleCloudPlatform/python-docs-samples#2677)

* automl: add natural language sentiment analysis ga samples

* Add links to documentation

* Update tests to use centralized project

* Fix environment variable, make translate test less flaky

* fix region tag typo [(#2731)](GoogleCloudPlatform/python-docs-samples#2731)

* Migrate published samples [(#2759)](GoogleCloudPlatform/python-docs-samples#2759)

Migrate from tmp-generated-samples branch 615c08e
Remove boilerplate
Update copyright date
Blacken
Remove unused imports
Shorten docstrings
Remove CLI
Set defaults in function definition
Add link to supported types guide
Inline function arguments
Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>

* translate v3 snippets [(#2745)](GoogleCloudPlatform/python-docs-samples#2745)

* translate text v3

* added translate text with glossary snippets

* finished glossary tests

* removed overlapping files

* added encoding tag

* added more descriptive docs and broke down tests

* Update translate/cloud-client/translate_v3_create_glossary.py

Co-Authored-By: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>

* Update translate/cloud-client/translate_v3_create_glossary.py

Co-Authored-By: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>

* fixed test for translate with glossary

* fixed lint

Co-authored-by: Michelle Casbon <texasmichelle@users.noreply.github.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>

* Translate: migrate published samples [(#2768)](GoogleCloudPlatform/python-docs-samples#2768)

Migrate from tmp-generated-samples branch 615c08e
Remove boilerplate
Update copyright date
Blacken
Remove unused imports

* Translate: migrate published glossaries samples [(#2769)](GoogleCloudPlatform/python-docs-samples#2769)

Migrate from tmp-generated-samples branch fef998b
Remove boilerplate
Update copyright date
Blacken
Remove unused imports
Shorten docstrings
Remove CLI

* translate: increase timeout [(#2937)](GoogleCloudPlatform/python-docs-samples#2937)

* Translate: migrate published  v3 translate batch samples [(#2914)](GoogleCloudPlatform/python-docs-samples#2914)

* Translate: migrate published b v3 tch samples

* added missing requirements

* extended wait time

* inlined some vals and specified input and output

* added link to supported file types & modified default values of input uri

* fixed small nit

* chore(deps): update dependency google-cloud-storage to v1.26.0 [(#3046)](GoogleCloudPlatform/python-docs-samples#3046)

* chore(deps): update dependency google-cloud-storage to v1.26.0

* chore(deps): specify dependencies by python version

* chore: up other deps to try to remove errors

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Co-authored-by: Leah Cole <coleleah@google.com>

* chore(deps): update dependency google-cloud-translate to v1.7.0 [(#3084)](GoogleCloudPlatform/python-docs-samples#3084)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-translate](https://togithub.com/googleapis/python-translate) | minor | `==1.6.0` -> `==1.7.0` |

---

### Release Notes

<details>
<summary>googleapis/python-translate</summary>

### [`v1.7.0`](https://togithub.com/googleapis/python-translate/blob/master/CHANGELOG.md#&#8203;170)

[Compare Source](https://togithub.com/googleapis/python-translate/compare/v1.6.0...v1.7.0)

10-07-2019 14:57 PDT

##### Implementation Changes

-   Update docstrings, client confg (via synth). ([#&#8203;9411](https://togithub.com/googleapis/google-cloud-python/pull/9411))
-   Remove send / receive message size limit (via synth). ([#&#8203;8974](https://togithub.com/googleapis/google-cloud-python/pull/8974))

##### New Features

-   Add support for V3 of the API. ([#&#8203;9020](https://togithub.com/googleapis/google-cloud-python/pull/9020))
-   Make `parent` argument required for all client methods in v3beta1; add `labels` argument (via synth). ([#&#8203;9354](https://togithub.com/googleapis/google-cloud-python/pull/9354))
-   Add client options to translate_v2. ([#&#8203;8737](https://togithub.com/googleapis/google-cloud-python/pull/8737))

##### Dependencies

-   Bump minimum version for google-api-core to 1.14.0. ([#&#8203;8709](https://togithub.com/googleapis/google-cloud-python/pull/8709))

##### Documentation

-   Fix links to reference documentation. ([#&#8203;8884](https://togithub.com/googleapis/google-cloud-python/pull/8884))
-   Link to googleapis.dev documentation in READMEs. ([#&#8203;8705](https://togithub.com/googleapis/google-cloud-python/pull/8705))

##### Internal / Testing Changes

-   Update `ListGlossaries` method annotation (via synth)  ([#&#8203;9385](https://togithub.com/googleapis/google-cloud-python/pull/9385))
-   Fix intersphinx reference to requests ([#&#8203;9294](https://togithub.com/googleapis/google-cloud-python/pull/9294))
-   Remove CI for gh-pages, use googleapis.dev for api_core refs. ([#&#8203;9085](https://togithub.com/googleapis/google-cloud-python/pull/9085))
-   Update intersphinx mapping for requests. ([#&#8203;8805](https://togithub.com/googleapis/google-cloud-python/pull/8805))

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* translate: fix tests [(#3087)](GoogleCloudPlatform/python-docs-samples#3087)

Fix: GoogleCloudPlatform/python-docs-samples#3018

* translate-v3:  samples [(#3034)](GoogleCloudPlatform/python-docs-samples#3034)

* translate with custom model, get supported langs

* inlined small nit

* added encoding to model test

* added missing region tags and link to supported langs

* inlined text-to-translate

* directly inlined contents

* revert text-translate vars

* reversed inlined text params

* small nit

Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>

* chore(deps): update dependency google-cloud-vision to v0.42.0 [(#3170)](GoogleCloudPlatform/python-docs-samples#3170)

* chore(deps): update dependency google-cloud-texttospeech to v1 [(#3210)](GoogleCloudPlatform/python-docs-samples#3210)

Co-authored-by: gcf-merge-on-green[bot] <60162190+gcf-merge-on-green[bot]@users.noreply.github.com>

* chore(deps): update dependency google-cloud-translate to v2 [(#3211)](GoogleCloudPlatform/python-docs-samples#3211)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-translate](https://togithub.com/googleapis/python-translate) | major | `==1.7.0` -> `==2.0.1` |

---

### Release Notes

<details>
<summary>googleapis/python-translate</summary>

### [`v2.0.1`](https://togithub.com/googleapis/python-translate/blob/master/CHANGELOG.md#&#8203;201-httpswwwgithubcomgoogleapispython-translatecomparev200v201-2020-01-31)

[Compare Source](https://togithub.com/googleapis/python-translate/compare/v2.0.0...v2.0.1)

### [`v2.0.0`](https://togithub.com/googleapis/python-translate/blob/master/CHANGELOG.md#&#8203;200)

[Compare Source](https://togithub.com/googleapis/python-translate/compare/v1.7.0...v2.0.0)

10-23-2019 11:13 PDT

##### New Features

-   Make v3 the default client. ([#&#8203;9498](https://togithub.com/googleapis/google-cloud-python/pull/9498))

##### Internal / Testing Changes

-   Add VPC-SC system tests. ([#&#8203;9272](https://togithub.com/googleapis/google-cloud-python/pull/9272))

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* Simplify noxfile setup. [(#2806)](GoogleCloudPlatform/python-docs-samples#2806)

* chore(deps): update dependency requests to v2.23.0

* Simplify noxfile and add version control.

* Configure appengine/standard to only test Python 2.7.

* Update Kokokro configs to match noxfile.

* Add requirements-test to each folder.

* Remove Py2 versions from everything execept appengine/standard.

* Remove conftest.py.

* Remove appengine/standard/conftest.py

* Remove 'no-sucess-flaky-report' from pytest.ini.

* Add GAE SDK back to appengine/standard tests.

* Fix typo.

* Roll pytest to python 2 version.

* Add a bunch of testing requirements.

* Remove typo.

* Add appengine lib directory back in.

* Add some additional requirements.

* Fix issue with flake8 args.

* Even more requirements.

* Readd appengine conftest.py.

* Add a few more requirements.

* Even more Appengine requirements.

* Add webtest for appengine/standard/mailgun.

* Add some additional requirements.

* Add workaround for issue with mailjet-rest.

* Add responses for appengine/standard/mailjet.

Co-authored-by: Renovate Bot <bot@renovateapp.com>

* Update dependency google-cloud-vision to v1 [(#3227)](GoogleCloudPlatform/python-docs-samples#3227)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [google-cloud-vision](https://togithub.com/googleapis/python-vision) | major | `==0.42.0` -> `==1.0.0` |

---

### Release Notes

<details>
<summary>googleapis/python-vision</summary>

### [`v1.0.0`](https://togithub.com/googleapis/python-vision/blob/master/CHANGELOG.md#&#8203;100-httpswwwgithubcomgoogleapispython-visioncomparev0420v100-2020-02-28)

[Compare Source](https://togithub.com/googleapis/python-vision/compare/v0.42.0...v1.0.0)

##### Features

-   bump release status to GA ([#&#8203;11](https://www.github.com/googleapis/python-vision/issues/11)) ([2129bde](https://www.github.com/googleapis/python-vision/commit/2129bdedfa0dca85c5adc5350bff10d4a485df77))

</details>

---

### Renovate configuration

:date: **Schedule**: At any time (no schedule defined).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples).

* translate: make test generic [(#3386)](GoogleCloudPlatform/python-docs-samples#3386)

Fix: GoogleCloudPlatform/python-docs-samples#3381

* [translate] fix: mark some tests as flaky [(#3495)](GoogleCloudPlatform/python-docs-samples#3495)

tests which involve LRO.

fixes #2875

* testing: replace @flaky with @pytest.mark.flaky [(#3496)](GoogleCloudPlatform/python-docs-samples#3496)

* testing: replace @flaky with @pytest.mark.flaky

* lint

* mark few tests as flaky

that involves LRO polling.

* lint

* chore(deps): update dependency google-cloud-storage to v1.28.0 [(#3260)](GoogleCloudPlatform/python-docs-samples#3260)

Co-authored-by: Takashi Matsuo <tmatsuo@google.com>

* translate: fix glossary leak issue [(#3572)](GoogleCloudPlatform/python-docs-samples#3572)

* fix glossary leak issue

* removed try/catch from teardown methods, removed sample_ prefix from all other methods

* added specific exceptions to tests, added backoff tags to tests

* fixed the lint issues

* reordered imports

* moved backoff inside methd and removed Retry

* corrected import nit

* chore: some lint fixes [(#3751)](GoogleCloudPlatform/python-docs-samples#3751)

* chore: some lint fixes

* longer timeout, more retries

* disable detect_test.py::test_async_detect_document

* chore(deps): update dependency google-cloud-storage to v1.28.1 [(#3785)](GoogleCloudPlatform/python-docs-samples#3785)

* chore(deps): update dependency google-cloud-storage to v1.28.1

* [asset] testing: use uuid instead of time

Co-authored-by: Takashi Matsuo <tmatsuo@google.com>

* [translate] fix hybrid glossaries tests [(#3936)](GoogleCloudPlatform/python-docs-samples#3936)

* testing: start using btlr [(#3959)](GoogleCloudPlatform/python-docs-samples#3959)

* testing: start using btlr

The binary is at gs://cloud-devrel-kokoro-resources/btlr/v0.0.1/btlr

* add period after DIFF_FROM

* use array for btlr args

* fix websocket tests

* add debug message

* wait longer for the server to spin up

* dlp: bump the wait timeout to 10 minutes

* [run] copy noxfile.py to child directory to avoid gcloud issue

* [iam] fix: only display description when the key exists

* use uuid4 instead of uuid1

* [iot] testing: use the same format for registry id

* Stop asserting Out of memory not in the output

* fix missing imports

* [dns] testing: more retries with delay

* [dlp] testing: longer timeout

* use the max-concurrency flag

* use 30 workers

* [monitoring] use multiple projects

* [dlp] testing: longer timeout

* Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](GoogleCloudPlatform/python-docs-samples#4022)

* remove whitelist replace with allowlist [(#4050)](GoogleCloudPlatform/python-docs-samples#4050)

* chore(deps): update dependency google-cloud-storage to v1.29.0 [(#4040)](GoogleCloudPlatform/python-docs-samples#4040)

* chore(deps): update dependency google-cloud-texttospeech to v2.1.0 [(#4147)](GoogleCloudPlatform/python-docs-samples#4147)

* testing(translate): parameterize the timeout [(#4247)](GoogleCloudPlatform/python-docs-samples#4247)

fixes #4239
(by specifying a longer timeout)

* chore(deps): update dependency pytest to v5.4.3 [(#4279)](GoogleCloudPlatform/python-docs-samples#4279)

* chore(deps): update dependency pytest to v5.4.3

* specify pytest for python 2 in appengine

Co-authored-by: Leah Cole <coleleah@google.com>

* Update dependency flaky to v3.7.0 [(#4300)](GoogleCloudPlatform/python-docs-samples#4300)

* testing(translate): bump the timeout for operations [(#4258)](GoogleCloudPlatform/python-docs-samples#4258)

fixes #4220

* chore: update templates

* chore: narrows samples CODEOWNERS to .py only

* chore: wip migration to microgenerator
client, units, docs complete

* feat!: move API to python microgenerator

* docs: readmegen updates

* chore: add build config for docs-presubmit

* chore: rm protos

* chore: uses PROJECT_ID env var in system test

* chore: clarifies examples in migration guide

* chore: adds explicit variable

Co-authored-by: Jason Dobry <jmdobry@users.noreply.github.com>
Co-authored-by: Jon Wayne Parrott <jonwayne@google.com>
Co-authored-by: Puneith Kaul <puneith@users.noreply.github.com>
Co-authored-by: DPE bot <dpebot@google.com>
Co-authored-by: Gus Class <gguuss@gmail.com>
Co-authored-by: florencep <florenceperot@google.com>
Co-authored-by: Bill Prin <waprin@gmail.com>
Co-authored-by: michaelawyu <chenyumic@google.com>
Co-authored-by: Frank Natividad <frankyn@users.noreply.github.com>
Co-authored-by: Averi Kitsch <akitsch@google.com>
Co-authored-by: Charles Engelke <github@engelke.com>
Co-authored-by: Rebecca Taylor <remilytaylor@gmail.com>
Co-authored-by: Elizabeth Crowdus <elcrowdus@gmail.com>
Co-authored-by: Noah Negrey <nnegrey@users.noreply.github.com>
Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com>
Co-authored-by: Michelle Casbon <texasmichelle@users.noreply.github.com>
Co-authored-by: Mike <45373284+munkhuushmgl@users.noreply.github.com>
Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com>
Co-authored-by: WhiteSource Renovate <bot@renovateapp.com>
Co-authored-by: Leah Cole <coleleah@google.com>
Co-authored-by: gcf-merge-on-green[bot] <60162190+gcf-merge-on-green[bot]@users.noreply.github.com>
Co-authored-by: Takashi Matsuo <tmatsuo@google.com>
Co-authored-by: Bu Sun Kim <8822365+busunkim96@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants