Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bazel presubmit fails on googleads/v3 using 0.34.2 #631

Closed
busunkim96 opened this issue Oct 2, 2020 · 2 comments · Fixed by #664
Closed

Bazel presubmit fails on googleads/v3 using 0.34.2 #631

busunkim96 opened this issue Oct 2, 2020 · 2 comments · Fixed by #664
Assignees
@software-dov
Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@busunkim96
Copy link
Contributor

The bazel presubmit for googleapis/googleapis is failing with the latest generator version. See log.

ERROR: /tmpfs/src/piper/google3/third_party/googleapis/google/ads/googleads/v3/BUILD.bazel:255:1: //google/ads/googleads/v3:googleads_py_micro_gapic: `bazel-out/host/bin/external/com_google_protobuf/protoc --experimental_allow_proto3_optional --plugin=protoc-gen-python_gapic=bazel-out/host/bin/external/gapic_generator_python/gapic_plugin --python_gapic_out=bazel-out/k8-fastbuild/bin/google/ads/googleads/v3/googleads_py_micro_gapic.srcjar.zip --python_gapic_opt=old-naming,lazy-import,python-gapic-name=googleads,python-gapic-templates=ads-templates -Igoogle/ads/googleads/v3/common/ad_asset.proto=google/ads/googleads/v3/common/ad_asset.proto -Igoogle/ads/googleads/v3/common/ad_type_infos.proto=google/ads/googleads/v3/common/ad_type_infos.proto -
...
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/cli/generate_with_pandoc.py", line 9, in <module>
    generate.generate()
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__click_7_1_2/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__click_7_1_2/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__click_7_1_2/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__click_7_1_2/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/cli/generate.py", line 63, in generate
    res = generator.Generator(opts).get_response(api_schema, opts)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/generator/generator.py", line 100, in get_response
    template_name, api_schema=api_schema, opts=opts)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/generator/generator.py", line 247, in _render_template
    template_name, api_schema=subpackage, opts=opts
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/generator/generator.py", line 285, in _render_template
    opts=opts,
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/generator/generator.py", line 312, in _get_file
    self._env.get_template(template_name).render(
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__Jinja2_2_11_2/jinja2/environment.py", line 883, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__Jinja2_2_11_2/jinja2/environment.py", line 857, in _load_template
    template = self.loader.load(self, name, globals)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__Jinja2_2_11_2/jinja2/loaders.py", line 127, in load
    code = environment.compile(source, name, filename)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__Jinja2_2_11_2/jinja2/environment.py", line 638, in compile
    self.handle_exception(source=source_hint)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__Jinja2_2_11_2/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__Jinja2_2_11_2/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/3696/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/ads-templates/%namespace/%name/%version/%sub/services/%service/transports/base.py.j2", line 123, in template
    '{{ service.name }}Transport',
jinja2.exceptions.TemplateSyntaxError: expected token ',', got '{'
--python_gapic_out: protoc-gen-python_gapic: Plugin failed with status code 1.
INFO: Elapsed time: 352.220s, Critical Path: 128.67s
INFO: 3871 processes: 3754 linux-sandbox, 117 worker.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

File is

gapic-generator-python/gapic/ads-templates/%namespace/%name/%version/%sub/services/%service/transports/base.py.j2

Line 123 in f7b1164

'{{ service.name }}Transport',
.
@yoshi-automation yoshi-automation added the triage me I really want to be triaged. label Oct 2, 2020
@software-dov software-dov added priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. and removed triage me I really want to be triaged. labels Oct 6, 2020
@busunkim96
Copy link
Contributor Author

Still fails with 0.35.3, but on a different API and bit of the code.

out/host/bin/external/com_google_protobuf/protoc --experimental_allow_proto3_optional '--plugin=protoc-gen-python_gapic=bazel-out/host/bin/external/gapic_generator_python/gapic_plugin' ... (remaining 33 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
google/cloud/securitycenter/v1beta1/security_marks.proto:20:1: warning: Import google/api/annotations.proto is unused.
google/cloud/securitycenter/v1beta1/asset.proto:24:1: warning: Import google/api/annotations.proto is unused.
google/cloud/securitycenter/v1beta1/finding.proto:19:1: warning: Import google/api/annotations.proto is unused.
google/cloud/securitycenter/v1beta1/organization_settings.proto:20:1: warning: Import google/api/annotations.proto is unused.
google/cloud/securitycenter/v1beta1/run_asset_discovery_response.proto:20:1: warning: Import google/api/annotations.proto is unused.
google/cloud/securitycenter/v1beta1/source.proto:20:1: warning: Import google/api/annotations.proto is unused.
google/cloud/securitycenter/v1beta1/securitycenter_service.proto:32:1: warning: Import google/protobuf/empty.proto is unused.
Traceback (most recent call last):
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/cli/generate_with_pandoc.py", line 9, in <module>
    generate.generate()
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__click_7_1_2/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__click_7_1_2/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__click_7_1_2/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__click_7_1_2/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/cli/generate.py", line 63, in generate
    res = generator.Generator(opts).get_response(api_schema, opts)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/generator/generator.py", line 100, in get_response
    template_name, api_schema=api_schema, opts=opts)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/generator/generator.py", line 285, in _render_template
    opts=opts,
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/generator/generator.py", line 313, in _get_file
    api=api_schema, opts=opts, **context
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__Jinja2_2_11_2/jinja2/environment.py", line 1090, in render
    self.environment.handle_exception()
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__Jinja2_2_11_2/jinja2/environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__Jinja2_2_11_2/jinja2/_compat.py", line 28, in reraise
    raise value.with_traceback(tb)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/templates/%namespace/%name_%version/%sub/services/%service/async_client.py.j2", line 1, in top-level template code
    {% extends '_base.py.j2' %}
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/templates/_base.py.j2", line 5, in top-level template code
    {%- block content %}
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/templates/%namespace/%name_%version/%sub/services/%service/async_client.py.j2", line 41, in block "content"
    {% for message in service.resource_messages|sort(attribute="resource_type") -%}
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python_pip_deps_pypi__Jinja2_2_11_2/jinja2/environment.py", line 471, in getattr
    return getattr(obj, attribute)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/utils/cache.py", line 39, in inner
    self._cached_values[fx.__name__] = fx(self)
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/schema/wrappers.py", line 1078, in resource_messages
    for method in self.methods.values()
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/schema/wrappers.py", line 1086, in <genexpr>
    method.lro.response_type if method.lro else method.output
  File "/tmpfs/tools/bazel_root/9c17b202d88c490a45ea631b0d45ecee/sandbox/linux-sandbox/5554/execroot/com_google_googleapis/bazel-out/host/bin/external/gapic_generator_python/gapic_plugin.runfiles/gapic_generator_python/gapic/schema/wrappers.py", line 1074, in gen_indirect_resources_used
    yield self.visible_resources[resource_type]
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/collections/__init__.py", line 885, in __getitem__
    return self.__missing__(key)            # support subclasses that define __missing__
  File "/home/kbuilder/.pyenv/versions/3.6.1/lib/python3.6/collections/__init__.py", line 877, in __missing__
    raise KeyError(key)
KeyError: '*'

@software-dov
Copy link
Contributor

I just ran into this issue myself while doing local development. Fix is inbound.

@software-dov software-dov mentioned this issue Oct 15, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@software-dov software-dov

Labels
priority: p2 Moderately-important priority. Fix may not be included in next release. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: minor typo in ads template software-dov/gapic-generator-python
3 participants
@software-dov @busunkim96 @yoshi-automation