Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Relax dependencies restriction for the gRPC libraries #2140

Merged

Conversation

tenzen-y
Copy link
Member

What this PR does / why we need it:
I relaxed the version restrictions on dependencies in the Python SDK.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #2139

Checklist:

  • Docs included if any changes are user facing

@google-oss-prow
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tenzen-y

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@tenzen-y
Copy link
Member Author

/assign @andreyvelich @johnugeorge

cc: @ryandawsonuk

Comment on lines 25 to 26
"grpcio>=1.41.1",
"protobuf>=3.19.5",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating this @tenzen-y!
Since we don't have proper tests in SDK to verify the GRPC lib functionality yet, can we update the versions in other Katib services as well. For example, tf event collector and other Suggestion services.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right.
I will change the versions in this PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@tenzen-y tenzen-y force-pushed the change-sdk-version-restriction branch from 87f161c to 1bff1c0 Compare March 31, 2023 17:35
@google-oss-prow google-oss-prow bot added size/M and removed size/XS labels Mar 31, 2023
@tenzen-y tenzen-y changed the title Relax dependencies restriction for Python SDK Relax dependencies restriction for the gRPC libraries Mar 31, 2023
Signed-off-by: Yuki Iwai <yuki.iwai.tz@gmail.com>
@tenzen-y tenzen-y force-pushed the change-sdk-version-restriction branch from 1bff1c0 to 3904581 Compare March 31, 2023 18:22
@tenzen-y
Copy link
Member Author

It seems that we must use the protobuf version lower or equal to v3.20.

Traceback (most recent call last):
File "run-e2e-experiment.py", line 6, in
from kubeflow.katib import ApiClient, KatibClient, models
File "/home/runner/work/katib/katib/sdk/python/v1beta1/kubeflow/katib/init.py", line 73, in
from kubeflow.katib.api.katib_client import KatibClient
File "/home/runner/work/katib/katib/sdk/python/v1beta1/kubeflow/katib/api/katib_client.py", line 27, in
import kubeflow.katib.katib_api_pb2 as katib_api_pb2
File "/home/runner/work/katib/katib/sdk/python/v1beta1/kubeflow/katib/katib_api_pb2.py", line 35, in
type=None),
File "/opt/hostedtoolcache/Python/3.7.16/x64/lib/python3.7/site-packages/google/protobuf/descriptor.py", line 796, in new
_message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:

  1. Downgrade the protobuf package to 3.20.x or lower.
  2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

https://github.com/kubeflow/katib/actions/runs/4577679497/jobs/8083373344?pr=2140#step:4:10112

@tenzen-y
Copy link
Member Author

Maybe, we must regenerate codes using a newer protoc in follow-up PRs.

@andreyvelich
Copy link
Member

@tenzen-y @johnugeorge We currently, use znly/protoc image to update our proto APIs..
It seems that this images have not been updated for 4 years: https://hub.docker.com/r/znly/protoc/tags.
Should we consider alternative ?

@tenzen-y
Copy link
Member Author

tenzen-y commented Apr 3, 2023

Should we consider alternative ?

@andreyvelich I agree. However, I think we can do that in another PR. wdyt?

@andreyvelich
Copy link
Member

@tenzen-y Sure, please can you create an issue to track it ?
/lgtm
/assign @johnugeorge

@google-oss-prow google-oss-prow bot added the lgtm label Apr 3, 2023
@google-oss-prow google-oss-prow bot merged commit 1d3ab57 into kubeflow:master Apr 3, 2023
@tenzen-y tenzen-y deleted the change-sdk-version-restriction branch April 3, 2023 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

katib python sdk incompatible pinned version
3 participants