Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Project requires an old version of Protobuf #2328

Open
wingdagger opened this issue Apr 24, 2024 · 8 comments
Open

Project requires an old version of Protobuf #2328

wingdagger opened this issue Apr 24, 2024 · 8 comments
Labels
pending on user response Waiting for more information or validation from user

Comments

@wingdagger
Copy link

The following setup.py file requires version ~=3.20 of protobuf. This is a very old release. Protobuf is currently on version 5.26.1. This constraint of ~=3.20 is causing conflicts when I generate my python dependencies. Can you please allow for newer versions of protobuf? Thank you.

https://github.com/onnx/tensorflow-onnx/blob/ed022438576ca7a351c3858105439f4f25c35219/setup.py#L85C95-L85C111
@fatcat-z
Copy link
Collaborator

Thanks for your comments..

At this moment, tf2onnx is still designed to support older TensorFlow versions as many as possible and some of them could not work on a version above 3.20.x. In addition, couple of dependencies of tf2onnx also required the protobuf version ~= 3.20.0, so we can't upgrade its version in setup.py now.

@fatcat-z fatcat-z added the pending on user response Waiting for more information or validation from user label Apr 26, 2024
@wingdagger
Copy link
Author

Thank you for the fast response. Shouldn't tensorflow indicate the prereq version of protobuf that it requires. tf2onnx should not need to specify versions required by another project, right?

@yan12125
Copy link
Contributor

Shouldn't tensorflow indicate the prereq version of protobuf that it requires.

Apparently only tensorflow > 2.10 does that. That's probably why many tests fail with older tensorflow versions in #2327.

@yan12125
Copy link
Contributor

Here is an idea: Python 3.11 is supported since tensorflow 2.12. I think it's safe to say on Python >= 3.11, tensorflow already properly specifies compatible protobuf versions. How about something like this?

install_requires=[
    # (other dependencies)
    'protobuf~=3.20; python_version < "3.11"',
    'protobuf; python_version >= "3.11"',
]

In addition, couple of dependencies of tf2onnx also required the protobuf version ~= 3.20.0

I didn't check other dependencies. Such a trick may apply as well.

@fatcat-z fatcat-z mentioned this issue May 31, 2024
Closed
@mjm522
Copy link

mjm522 commented Dec 19, 2024

Could this be done on the setup.py. Currently the version 1.61.1 is not compatible with bigquery or any gcp package that requires protobuf > 4 Though we could fork and it and do it, just wanted know the possibility of updating it on the main? I understand from the comments that it does work with protobuf > 4

andyatmiami added a commit to andyatmiami/notebooks that referenced this issue Dec 20, 2024
@andyatmiami
feat(deps): add kubeflow-training to workbench images
08e327e 
This commit adds `kubeflow-training[huggingface]` to the following workbench images:
- `./jupyter/datascience/ubi9-python-3.11`
- `./jupyter/pytorch/ubi9-python-3.11`
- `./jupyter/rocm/pytorch/ubi9-python-3.11`
- `./jupyter/trustyai/ubi9-python-3.11`
- `./codeserver/ubi9-python-3.11`

This outcome comes with a slew of caveats and disclaimers:
- Due to a dependency conflict, `codeflare-sdk==0.24.3` was **also** added to the following workbench images.
    - `./jupyter/datascience/ubi9-python-3.11`
    - `./jupyter/pytorch/ubi9-python-3.11`
    - `./jupyter/rocm/pytorch/ubi9-python-3.11`
    - `./jupyter/trustyai/ubi9-python-3.11`
- ⚠️ In what may be a "controversial" decision, `codeflare-sdk` was **NOT** updated on other workbench images.  Since `0.24.3` was a "one-off" release to unblock the `kubeflow-training` inclusion - the thought process here is that normal "sync" procedures on the next official release will standardize the `codeflare-sdk` dependency across all workbench images.  This allows us to restrict the testing effort of this commit.
- `jupyter/minmal/ubi9-python-3.11` was deliberately excluded from receiving `kubeflow-training` per discussions with team.
- Due to dependency conflicts discovered `tensorflow`-based workbench images,`kubeflow-training` has not been added to those workbench images at this time.  This decision was agreed to by affect stakeholders.  Core blocking issue can be seen here:
    - onnx/tensorflow-onnx#2328
- Due to a dependency conflict, `transformers = "==4.38.0"` was **also** added to the`./jupyter/trustyai/ubi9-python-3.11` workbench image after discussion with the developer that last worked on the `trustyai` image.  While it certainly must be tested, there was no strict requirement that necessitated pinning the `transformers` dependency to `4.36.2` - and the `huggingface` `extras` now introduces a `4.38.0` constraint for `transformers`.

Related-to: https://issues.redhat.com/browse/RHOAIENG-12822
@andyatmiami andyatmiami mentioned this issue Dec 20, 2024
Merged
3 tasks
andyatmiami added a commit to andyatmiami/notebooks that referenced this issue Dec 20, 2024
@andyatmiami
feat(deps): add kubeflow-training to workbench images
16b5c6d 
This commit adds `kubeflow-training[huggingface]` to the following workbench images:
- `./jupyter/datascience/ubi9-python-3.11`
- `./jupyter/pytorch/ubi9-python-3.11`
- `./jupyter/rocm/pytorch/ubi9-python-3.11`
- `./jupyter/trustyai/ubi9-python-3.11`
- `./codeserver/ubi9-python-3.11`

This outcome comes with a slew of caveats and disclaimers:
- Due to a dependency conflict, `codeflare-sdk==0.24.3` was **also** added to the following workbench images.
    - `./jupyter/datascience/ubi9-python-3.11`
    - `./jupyter/pytorch/ubi9-python-3.11`
    - `./jupyter/rocm/pytorch/ubi9-python-3.11`
    - `./jupyter/trustyai/ubi9-python-3.11`
- ⚠️ In what may be a "controversial" decision, `codeflare-sdk` was **NOT** updated on other workbench images.  Since `0.24.3` was a "one-off" release to unblock the `kubeflow-training` inclusion - the thought process here is that normal "sync" procedures on the next official release will standardize the `codeflare-sdk` dependency across all workbench images.  This allows us to restrict the testing effort of this commit.
- `jupyter/minmal/ubi9-python-3.11` was deliberately excluded from receiving `kubeflow-training` per discussions with team.
- Due to dependency conflicts discovered `tensorflow`-based workbench images,`kubeflow-training` has not been added to those workbench images at this time.  This decision was agreed to by affect stakeholders.  Core blocking issue can be seen here:
    - onnx/tensorflow-onnx#2328
- Due to a dependency conflict, `transformers = "==4.38.0"` was **also** added to the`./jupyter/trustyai/ubi9-python-3.11` workbench image after discussion with the developer that last worked on the `trustyai` image.  While it certainly must be tested, there was no strict requirement that necessitated pinning the `transformers` dependency to `4.36.2` - and the `huggingface` `extras` now introduces a `4.38.0` constraint for `transformers`.

Related-to: https://issues.redhat.com/browse/RHOAIENG-12822
andyatmiami added a commit to andyatmiami/notebooks that referenced this issue Dec 20, 2024
@andyatmiami
feat(deps): add kubeflow-training to workbench images
e09fa47 
This commit adds `kubeflow-training[huggingface]` to the following workbench images:
- `./jupyter/datascience/ubi9-python-3.11`
- `./jupyter/pytorch/ubi9-python-3.11`
- `./jupyter/rocm/pytorch/ubi9-python-3.11`
- `./jupyter/trustyai/ubi9-python-3.11`
- `./codeserver/ubi9-python-3.11`

This outcome comes with a slew of caveats and disclaimers:
- Due to a dependency conflict, `codeflare-sdk==0.24.3` was **also** added to the following workbench images.
    - `./jupyter/datascience/ubi9-python-3.11`
    - `./jupyter/pytorch/ubi9-python-3.11`
    - `./jupyter/rocm/pytorch/ubi9-python-3.11`
    - `./jupyter/trustyai/ubi9-python-3.11`
- ⚠️ In what may be a "controversial" decision, `codeflare-sdk` was **NOT** updated on other workbench images.  Since `0.24.3` was a "one-off" release to unblock the `kubeflow-training` inclusion - the thought process here is that normal "sync" procedures on the next official release will standardize the `codeflare-sdk` dependency across all workbench images.  This allows us to restrict the testing effort of this commit.
- `jupyter/minmal/ubi9-python-3.11` was deliberately excluded from receiving `kubeflow-training` per discussions with team.
- Due to dependency conflicts discovered `tensorflow`-based workbench images,`kubeflow-training` has not been added to those workbench images at this time.  This decision was agreed to by affect stakeholders.  Core blocking issue can be seen here:
    - onnx/tensorflow-onnx#2328
- Due to a dependency conflict, `transformers = "==4.38.0"` was **also** added to the`./jupyter/trustyai/ubi9-python-3.11` workbench image after discussion with the developer that last worked on the `trustyai` image.  While it certainly must be tested, there was no strict requirement that necessitated pinning the `transformers` dependency to `4.36.2` - and the `huggingface` `extras` now introduces a `4.38.0` constraint for `transformers`.

Related-to: https://issues.redhat.com/browse/RHOAIENG-12822
andyatmiami added a commit to andyatmiami/notebooks that referenced this issue Dec 20, 2024
@andyatmiami
feat(deps): add kubeflow-training to workbench images
429e683 
This commit adds `kubeflow-training[huggingface]` to the following workbench images:
- `./jupyter/datascience/ubi9-python-3.11`
- `./jupyter/pytorch/ubi9-python-3.11`
- `./jupyter/rocm/pytorch/ubi9-python-3.11`
- `./jupyter/trustyai/ubi9-python-3.11`
- `./codeserver/ubi9-python-3.11`

This outcome comes with a slew of caveats and disclaimers:
- Due to a dependency conflict, `codeflare-sdk==0.24.3` was **also** added to the following workbench images.
    - `./jupyter/datascience/ubi9-python-3.11`
    - `./jupyter/pytorch/ubi9-python-3.11`
    - `./jupyter/rocm/pytorch/ubi9-python-3.11`
    - `./jupyter/trustyai/ubi9-python-3.11`
- ⚠️ In what may be a "controversial" decision, `codeflare-sdk` was **NOT** updated on other workbench images.  Since `0.24.3` was a "one-off" release to unblock the `kubeflow-training` inclusion - the thought process here is that normal "sync" procedures on the next official release will standardize the `codeflare-sdk` dependency across all workbench images.  This allows us to restrict the testing effort of this commit.
- `jupyter/minmal/ubi9-python-3.11` was deliberately excluded from receiving `kubeflow-training` per discussions with team.
- Due to dependency conflicts discovered `tensorflow`-based workbench images,`kubeflow-training` has not been added to those workbench images at this time.  This decision was agreed to by affect stakeholders.  Core blocking issue can be seen here:
    - onnx/tensorflow-onnx#2328
- Due to a dependency conflict, `transformers = "==4.38.0"` was **also** added to the`./jupyter/trustyai/ubi9-python-3.11` workbench image after discussion with the developer that last worked on the `trustyai` image.  While it certainly must be tested, there was no strict requirement that necessitated pinning the `transformers` dependency to `4.36.2` - and the `huggingface` `extras` now introduces a `4.38.0` constraint for `transformers`.

Related-to: https://issues.redhat.com/browse/RHOAIENG-12822
andyatmiami added a commit to andyatmiami/notebooks that referenced this issue Dec 20, 2024
@andyatmiami
feat(deps): add kubeflow-training to workbench images
b2d3631 
This commit adds `kubeflow-training[huggingface]` to the following workbench images:
- `./jupyter/datascience/ubi9-python-3.11`
- `./jupyter/pytorch/ubi9-python-3.11`
- `./jupyter/rocm/pytorch/ubi9-python-3.11`
- `./jupyter/trustyai/ubi9-python-3.11`
- `./codeserver/ubi9-python-3.11`

This outcome comes with a slew of caveats and disclaimers:
- Due to a dependency conflict, `codeflare-sdk==0.24.3` was **also** added to the following workbench images.
    - `./jupyter/datascience/ubi9-python-3.11`
    - `./jupyter/pytorch/ubi9-python-3.11`
    - `./jupyter/rocm/pytorch/ubi9-python-3.11`
    - `./jupyter/trustyai/ubi9-python-3.11`
- ⚠️ In what may be a "controversial" decision, `codeflare-sdk` was **NOT** updated on other workbench images.  Since `0.24.3` was a "one-off" release to unblock the `kubeflow-training` inclusion - the thought process here is that normal "sync" procedures on the next official release will standardize the `codeflare-sdk` dependency across all workbench images.  This allows us to restrict the testing effort of this commit.
- `jupyter/minmal/ubi9-python-3.11` was deliberately excluded from receiving `kubeflow-training` per discussions with team.
- Due to dependency conflicts discovered `tensorflow`-based workbench images,`kubeflow-training` has not been added to those workbench images at this time.  This decision was agreed to by affect stakeholders.  Core blocking issue can be seen here:
    - onnx/tensorflow-onnx#2328
- Due to a dependency conflict, `transformers = "==4.38.0"` was **also** added to the`./jupyter/trustyai/ubi9-python-3.11` workbench image after discussion with the developer that last worked on the `trustyai` image.  While it certainly must be tested, there was no strict requirement that necessitated pinning the `transformers` dependency to `4.36.2` - and the `huggingface` `extras` now introduces a `4.38.0` constraint for `transformers`.

Related-to: https://issues.redhat.com/browse/RHOAIENG-12822
andyatmiami added a commit to andyatmiami/notebooks that referenced this issue Dec 29, 2024
@andyatmiami
feat(deps): add kubeflow-training to workbench images
70c151d 
This commit adds `kubeflow-training[huggingface]` to the following workbench images:
- `./jupyter/datascience/ubi9-python-3.11`
- `./jupyter/pytorch/ubi9-python-3.11`
- `./jupyter/rocm/pytorch/ubi9-python-3.11`
- `./jupyter/trustyai/ubi9-python-3.11`
- `./codeserver/ubi9-python-3.11`

This outcome comes with a slew of caveats and disclaimers:
- Due to a dependency conflict, `codeflare-sdk==0.24.3` was **also** added to the following workbench images.
    - `./jupyter/datascience/ubi9-python-3.11`
    - `./jupyter/pytorch/ubi9-python-3.11`
    - `./jupyter/rocm/pytorch/ubi9-python-3.11`
    - `./jupyter/trustyai/ubi9-python-3.11`
- ⚠️ In what may be a "controversial" decision, `codeflare-sdk` was **NOT** updated on other workbench images.  Since `0.24.3` was a "one-off" release to unblock the `kubeflow-training` inclusion - the thought process here is that normal "sync" procedures on the next official release will standardize the `codeflare-sdk` dependency across all workbench images.  This allows us to restrict the testing effort of this commit.
- `jupyter/minmal/ubi9-python-3.11` was deliberately excluded from receiving `kubeflow-training` per discussions with team.
- Due to dependency conflicts discovered `tensorflow`-based workbench images,`kubeflow-training` has not been added to those workbench images at this time.  This decision was agreed to by affect stakeholders.  Core blocking issue can be seen here:
    - onnx/tensorflow-onnx#2328
- Due to a dependency conflict, `transformers = "==4.38.0"` was **also** added to the`./jupyter/trustyai/ubi9-python-3.11` workbench image after discussion with the developer that last worked on the `trustyai` image.  While it certainly must be tested, there was no strict requirement that necessitated pinning the `transformers` dependency to `4.36.2` - and the `huggingface` `extras` now introduces a `4.38.0` constraint for `transformers`.

Related-to: https://issues.redhat.com/browse/RHOAIENG-12822
andyatmiami added a commit to andyatmiami/notebooks that referenced this issue Jan 2, 2025
@andyatmiami
feat(deps): add kubeflow-training to workbench images
f9958e4 
This commit adds `kubeflow-training[huggingface]` to the following workbench images:
- `./jupyter/datascience/ubi9-python-3.11`
- `./jupyter/pytorch/ubi9-python-3.11`
- `./jupyter/rocm/pytorch/ubi9-python-3.11`
- `./jupyter/trustyai/ubi9-python-3.11`
- `./codeserver/ubi9-python-3.11`

This outcome comes with a slew of caveats and disclaimers:
- Due to a dependency conflict, `codeflare-sdk==0.24.3` was **also** added to the following workbench images.
    - `./jupyter/datascience/ubi9-python-3.11`
    - `./jupyter/pytorch/ubi9-python-3.11`
    - `./jupyter/rocm/pytorch/ubi9-python-3.11`
    - `./jupyter/trustyai/ubi9-python-3.11`
- ⚠️ In what may be a "controversial" decision, `codeflare-sdk` was **NOT** updated on other workbench images.  Since `0.24.3` was a "one-off" release to unblock the `kubeflow-training` inclusion - the thought process here is that normal "sync" procedures on the next official release will standardize the `codeflare-sdk` dependency across all workbench images.  This allows us to restrict the testing effort of this commit.
- `jupyter/minmal/ubi9-python-3.11` was deliberately excluded from receiving `kubeflow-training` per discussions with team.
- Due to dependency conflicts discovered `tensorflow`-based workbench images,`kubeflow-training` has not been added to those workbench images at this time.  This decision was agreed to by affect stakeholders.  Core blocking issue can be seen here:
    - onnx/tensorflow-onnx#2328
- Due to a dependency conflict, `transformers = "==4.38.0"` was **also** added to the`./jupyter/trustyai/ubi9-python-3.11` workbench image after discussion with the developer that last worked on the `trustyai` image.  While it certainly must be tested, there was no strict requirement that necessitated pinning the `transformers` dependency to `4.36.2` - and the `huggingface` `extras` now introduces a `4.38.0` constraint for `transformers`.

Related-to: https://issues.redhat.com/browse/RHOAIENG-12822
@OvervCW
Copy link

OvervCW commented Jan 8, 2025 •
edited
Loading

Any news about this? We currently have to maintain a fork with a single line change because of this requirement.

And when will this package officially support Python 3.11 and later?

@dmal-msft
Copy link

Any news about this? We currently have to maintain a fork with a single line change because of this requirement.

And when will this package officially support Python 3.11 and later?
Any updated on this?
We are interested about the python upgrade as well

@jiridanek
Copy link

jiridanek commented Jan 22, 2025 •
edited
Loading

Related to

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
pending on user response Waiting for more information or validation from user
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@jiridanek @yan12125 @wingdagger @mjm522 @fatcat-z @OvervCW @dmal-msft